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WARGAMING  EFFECTIVENESS:  ITS  CONCEPTUALIZATION  AND  ASSESSMENT 

EXECUTIVE  SUMMARY _ 


Research  Requirement: 

Battalions  and  brigades  are  experiencing  substantial  change  in  structure  and  identity  as 
they  transform  to  modular  units  of  action.  Among  other  things,  this  change  has  implications  for 
how  to  apply  the  military  decision-making  process  to  planning  unconventional  missions  using 
new  types  of  units.  Wargaming  arguably  is  the  most  important  collective  activity  occurring 
during  operational  planning.  The  need  to  understand,  develop,  and  support  battle  staff 
wargaming  has  never  been  greater  than  it  is  now,  when  innovative  problem  solving  is  absolutely 
critical  to  success. 

Developing  and  supporting  the  collective  activity  involved  in  wargaming  requires  that  the 
outcomes  of  wargaming  be  well  defined,  that  the  processes  by  which  successful  wargaming 
occurs  be  well  understood,  and  that  the  causes  for  ineffective  wargaming  be  recognized. 
Assessment  lies  at  the  intersection  of  understanding  wargaming  and  actually  improving  it 
through  training  or  command  and  control  system  design.  Without  assessment,  understanding 
wargaming  cannot  be  linked  to  performance  enhancement. 

The  collective  activity  involved  in  effective  wargaming,  however,  is  not  well  understood, 
and  its  assessment  is  largely  unexplored.  The  understanding  of  wargaming  effectiveness  must  go 
beyond  current  conceptualizations  to  illuminate  the  constructs  that  comprise  effective 
wargaming.  Assessment  of  wargaming  effectiveness  must  go  beyond  current  state-of-the-art  to 
capture  these  constructs  in  a  reliable,  valid  manner. 

The  purpose  of  the  present  research  was  to  determine  the  constructs  that  comprise 
effective  wargaming  and  to  explore  methods  for  assessing  these  constructs.  The  focus  of  the 
research  was  wargaming  as  conducted  by  the  staffs  of  combined  arms  battalion  task  forces.  The 
goal  was  to  provide  direction  for  conducting  wargaming  assessment  for  the  purposes  of  tracking 
training  progress  and/or  evaluating  the  effects  of  technological  intervention  on  wargaming 
effectiveness. 

Procedure: 

Cognitive  task  analysis  was  used  to  develop  a  conceptual  framework  for  understanding 
the  knowledge,  skills,  and  other  attributes  that  comprise  the  individual  and  team-related 
determinants,  processes,  and  outcomes  of  effective  wargaming.  This  framework  was  used  as  the 
basis  for  designing  and  implementing  assessments  of  individual  and  team  wargaming 
competence.  The  validity  and  feasibility  of  these  assessments  was  explored  by  administering  the 
assessments  during  Armor  Captain’s  Career  Course  (AC3)  training  between  November  2004  and 
September  2005.  Six  independent  groups  of  students  participated  in  the  research. 
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Findings: 

This  initial  exploration  of  wargaming  assessment  methods  indicates  that  assessments 
derived  from  the  wargaming  conceptual  framework  can  be  feasible  to  administer  and  be  reliable 
and  valid  assessments  of  their  related  psychological  constructs.  In  addition,  these  assessments 
are  informative  regarding  the  possible  situational  factors  that  influence  wargaming  performance, 
including  the  number  of  officers  involved,  the  ease  of  communications,  and  the  adequacy  of 
supporting  equipment.  Further  validation  of  wargaming  effectiveness  assessments  requires  a 
combined  approach  in  which  assessments  are  designed  based  on  an  understanding  of  wargaming, 
tested  in  a  controlled  experimental  environment  where  specific  conditions  can  be  manipulated, 
and  validated  in  an  operational  setting. 

Utilization  and  Dissemination  of  Findings: 

The  present  research  produces  a  more  in-depth  understanding  of  wargaming  and  its 
effectiveness  assessment  than  previously  has  been  accomplished.  Several  psychological 
constructs  comprising  wargaming  determinants  (individual  and  team-related),  processes,  and 
outcomes  have  been  identified.  This  work  therefore  sets  the  stage  for  more  systematic 
assessment  of  wargaming  effectiveness  and  diagnosis  of  shortfalls  in  wargaming  performance. 

If  extended  and  applied,  this  work  could  escort  operations  command  and  control  teams  into  the 
future  through  a  better  understanding  of  how  to  develop  and  support  their  collective  mission 
planning  competence. 
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Introduction 


As  the  U.S.  Army  goes  through  its  greatest  transformation  since  the  1960’s  (Smith,  2005) 
(a  transformation  necessitated  by  shifts  in  the  nature  of  warfare  and  the  fundamental  mission  of 
the  Army)  change  over  the  next  several  years  will  effect  every  echelon  in  the  organization. 
Battalions  and  brigades  are  experiencing  the  greatest  change  in  structure  and  identity  as  they 
transform  to  modular  units  of  action.  Among  other  things,  transformation  has  implications  for 
unit  employment  tactics  and  therefore  the  “art”  of  mission  command  \FM6-0;  U.S.  Department 
of  the  Army  (DA),  2003].  Battalion  and  brigade  commanders  and  their  staffs  must  leam  how  to 
apply  doctrinal  concepts,  such  as  the  military  decision-making  process  in  new  ways  to  plan 
unconventional  missions  and  counter  asymmetric  enemies.  Moreover,  they  must  apply  these 
concepts  using  new  technology,  specifically  networked  digital  command  and  control  systems. 
The  need  to  understand,  develop,  and  support  the  collective  activity  that  comprises  effective 
operational  (i.e.,  battalion-  and  brigade-level)  planning  and  decision-making  has  never  been 
greater  than  it  is  in  the  contemporary  operating  environment,  where  innovative  problem  solving 
is  absolutely  critical  to  success. 

Wargaming,  the  analysis  of  potential  courses  of  action  for  their  feasibility,  risk,  and 
likelihood  of  success  (FM  5-0;  DA,  2002),  arguably  is  the  most  important  collective  activity 
occurring  during  operational  planning.  Effective  wargaming  not  only  accomplishes  a  systematic 
evaluation  of  candidate  mission  plans,  but  also  serves  to  align  the  battalion  or  brigade 
commander’s  visualization  of  an  upcoming  engagement  with  that  of  his  staff.  This  shared 
visualization  is  what  enables  staff  officers  to  anticipate  the  commander’s  decision-making  needs 
in  the  midst  of  an  engagement  and  therefore  provide  him  timely,  relevant  information  or  even 
make  decisions  in  his  absence.  In  addition,  effective  wargaming  enables  staff  officers  to  develop 
integrated  mission  plans  that  synchronize  their  functional  areas  to  achieve  decisive  effects. 
Mission  plans  do  not  apply  fully  after  the  enemy  makes  his  first  unpredicted  action,  but  it  is 
through  the  wargaming  process  that  the  commander  and  his  staff  internalize  the  overall  purpose 
and  goals  of  the  mission  as  well  as  explore  the  implications  of  the  operational  environment  for 
action. 


Developing  and  supporting  the  collective  activity  involved  in  wargaming  requires  that  the 
outcomes  of  wargaming  be  well  defined,  that  the  processes  by  which  successful  wargaming 
occurs  be  well  understood,  and  that  the  causes  for  ineffective  wargaming  be  recognized. 
Understanding  the  determinants  (individual  and  team-related),  processes,  and  outcomes  of 
wargaming  focuses  the  determination  of  training  objectives  or  identification  of  behavioral  targets 
for  technological  support.  Assessment  lies  at  the  intersection  of  understanding  wargaming  and 
actually  improving  it  through  training  or  command  and  control  system  design.  It  is  through 
assessment  that  progress  toward  training  objectives  can  be  tracked  and  facilitated.  It  is  also 
through  assessment  that  the  effects  of  technology  or  other  intervention  on  collective  activity  are 
revealed  and  evaluated.  Without  assessment,  understanding  wargaming  cannot  be  linked  to 
performance  enhancement. 

The  purpose  of  the  present  research  was  to  design,  develop,  and  evaluate  the  reliability 
and  feasibility  of  techniques  for  assessing  wargaming  effectiveness.  The  focus  of  this  research 
was  wargaming  as  conducted  by  the  staffs  of  combined  arms  battalion  task  forces  (CABTFs). 
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The  goal  was  to  provide  direction  for  conducting  wargaming  assessment  for  the  purposes  of 
tracking  training  progress  and/or  evaluating  the  effects  of  technological  intervention  on 
wargaming  effectiveness.  The  following  quote  from  Messick  (1994),  as  cited  in  Mislevy, 
Steinberg,  Breyer,  Almond,  and  Johnson  (1999)  precisely  summarizes  the  overall  approach  used. 

“[We]  would  begin  by  asking  what  complex  of  knowledge,  skills,  or  other 
attributes  should  be  assessed,  presumably  because  they  are  tied  to  explicit  or 
implicit  objectives  of  instruction  or  are  otherwise  valued  by  society.  Next,  what 
behaviors  or  performances  should  reveal  those  constructs  and  what  tasks  or 
situations  should  elicit  those  behaviors?  Thus,  the  nature  of  the  construct  guides 
the  selection  or  construction  of  relevant  [assessment]  tasks  as  well  as  the  rational 
development  of  scoring  criteria  and  rubrics,  (p.  17)” 

The  knowledge  skills,  or  other  attributes  comprising  the  collective  activity  involved  in 
effective  wargaming,  however,  are  not  well  understood.  Doctrine  ( FM5-0\  DA,  2002)  and  other 
sources  (Ford  &  Campbell,  1997;  Mullen,  Kemper,  Harrison,  &  Bartkoski,  1997)  state  the  tasks 
that  must  be  accomplished  for  wargaming  to  be  considered  successful,  but  do  not  identify  the 
determinants  (individual  and  team-related),  processes,  and  outcomes  of  wargaming.  If  the  tasks 
required  for  successful  wargaming  are  accomplished,  it  is  still  unknown  whether  the  desired 
outcome  of  wargaming  has  actually  been  achieved  (e.g.,  a  person  may  successfully  hammer  three 
nails  into  the  wall  and  set  a  shelf  upon  them,  but  the  shelf  may  be  crooked,  may  not  coordinate 
with  the  other  decor  in  the  room,  etc.).  If  these  tasks  are  not  accomplished,  it  is  unknown  as  to 
why  there  was  a  shortfall  in  performance. 

For  this  reason,  the  present  research  also  focused  on  developing  a  conceptual  framework 
for  understanding  knowledge,  skills,  and  other  attributes  that  comprise  the  determinants, 
processes,  and  outcomes  of  effective  wargaming.  This  framework  served  as  the  basis  for 
designing,  implementing,  and  evaluating  assessments  of  individual  and  team  wargaming 
competence.  The  reliability  and  feasibility  of  these  assessments  was  evaluated  through 
administration  during  Armor  Captain’s  Career  Course  (AC3)  training  occurring  between 
November  2004  and  September  2005.  This  final  report  presents  the  research  approach  in  detail, 
the  assessments  developed,  and  the  findings  of  their  evaluation.  For  the  convenience  of  the 
reader,  a  list  of  all  acronyms  is  provided  in  Appendix  A. 

A  Cognitive  Task  Analysis  of  Wargaming 

In  order  to  identify  the  knowledge,  skills,  and  other  attributes  that  should  be  assessed  to 
capture  the  wargaming  effectiveness  of  CABTF  staffs,  a  cognitive  task  analysis  of  wargaming 
was  conducted.  The  objective  was  to  identify  not  only  the  cognitive  activities  involved  in 
effective  wargaming,  but  also  the  behavioral  indicators  of  effective  performance  of  these 
activities.  For  this  reason,  the  approach  used  combined  applied  cognitive  task  analysis 
(described  in  Militello  &  Hutton,  1998)  with  evidence-centered  cognitive  task  analysis 
(described  in  Mislevy  et  al.,  1999).  This  approach  allowed  the  present  assessment  research  to  be 
based  on  a  conceptualization  of  effective  operational  behavior  in  terms  of  psychological 
constructs,  rather  than  on  an  a  priori  selection  of  psychological  constructs  that  were  force-fit  to  a 
poorly  understood  notion  of  operational  behavior.  Unlike  either  Militello,  and  Hutton  (1998)  or 
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Mislevy  et  al.  (1999),  the  present  task  analysis  focused  on  collective  as  well  as  individual 
cognitive  activities.  The  task  analysis  was  conducted  in  three  phases:  literature  review, 
discussion  with  subject  matter  experts  (SMEs),  and  observation  of  ongoing  wargaming. 

Literature  Review 

The  task  analysis  began  with  a  review  of  several  unclassified  Army  field  manuals  on 
mission  planning  and  tactics  [e.g.,  FM  5-0;  DA,  2002]  such  that  a  general  understanding  of  the 
wargaming  process  could  be  accomplished.  The  documentation  reviewed  described  the  purpose 
of  the  wargaming  process,  its  procedural  steps,  and  the  key  people  involved  and  their  role  in  the 
process.  Articles  in  the  Army  professional  literature  [e.g.,  Center  for  Army  Lessons  Learned 
(CALL),  1998],  as  well  as  a  variety  of  publicly  accessible  Government-sponsored  technical  and 
research  reports  (e.g.,  Mullen  et  al.,  1997),  monographs  (e.g.,  Glenn,  1996),  and  military  student 
thesis  (e.g.,  Crain,  1989),  helped  with  identifying  the  particularly  challenging  aspects  of 
wargaming.  These  aspects  are  successfully  conducted  by  expert  teams  of  wargamers  and 
unsuccessfully  by  others.  These  aspects  are  also  those  of  greatest  importance  to  ensuring  that  the 
wargame  achieves  its  purpose.  A  general  understanding  of  the  wargaming  process  and  its 
challenging  aspects  prepared  the  principal  investigator  to  interview  SMEs  for  more  in-depth 
information  not  readily  available  in  the  literature. 

Interviews  with  Subject  Matter  Experts 

Interviews  with  17  SMEs  were  conducted  to  better  understand  why  aspects  of  the 
wargaming  process  are  difficult  for  CABTF  staffs  and  to  determine  the  individual  and  team 
activities  that  comprise  effective  wargaming.  Interviewees  included  current  and  former 
observer/controllers  at  the  U.S.  Army  National  Training  Center,  active  and  retired  lieutenant 
colonels  with  battalion  command  experience,  instructors  at  the  U.S.  Army  Command  and 
General  Staff  College  and  at  the  U.S.  Army  Armor  School,  and  two  retired  generals.  All  of  these 
SMEs  had  experience  as  staff  officers  and  as  members  and/or  commanders  of  a  combined  arms 
unit.  The  interviews  conducted  were  not  structured  around  the  probe  questions  presented  in 
Militello  and  Hutton  (1998)  or  Mislevy  et  al.  (1999);  the  probes  used  by  these  researchers  to 
capture  decision-making  activity  were  not  relevant  to  analyzing  an  information-gathering  task 
such  as  wargaming.  Rather,  the  interviews  in  the  present  research  started  from  the  following 
general  questions:  (1)  What  makes  [a  particular  aspect  of  wargaming]  difficult?  (2)  What  do 
staff  officers  have  to  know  and/or  do  to  [perform  this  particular  aspect]  successfully?  (3)  What 
mistakes  do  less  experienced  staffs  typically  make  [when  performing  this  aspect]?  (4)  What  do 
these  mistakes  look  like?  (5)  What  does  it  look  like  when  [this  particular  aspect]  is  performed 
effectively? 

Observations 

As  a  supplement  to  SME  interviews  in  determining  the  indicators  of  wargaming 
effectiveness,  observations  were  conducted  of  ongoing  wargaming  instruction  and  of  wargaming 
conducted  during  a  high-fidelity  field  training  exercise.  The  wargaming  instruction  observed 
was  conducted  at  the  Armor  School.  Students  were  National  Guard  officers  and  regular  Army 
officers  participating  in  AC3.  Regular  Army  students  in  the  resident  version  of  AC3  wargamed 
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together  in  a  classroom  using  acetate  overlays  and  paper  maps.  National  Guard  students  in  the 
distance-learning  version  of  AC3  (AC3DL)  wargamed  while  geographically  distributed  using  a 
virtual  tactical  operations  center  (VTOC;  see  Sanders,  2002).  The  field  exercise  observed  was 
conducted  at  the  U.S.  Army  National  Training  Center.  Brigade  and  armor  task  force  planning 
was  observed  as  these  units  went  through  stability  and  security  operations  training.  All 
observations  were  conducted  in  collaboration  with  SMEs  who  provided  explanation  and  insights 
in  real  time. 


A  Framework  for  Conceptualizing  Wargaming  Effectiveness 

This  section  presents  the  findings  of  the  cognitive  task  analysis  in  the  form  of  a 
conceptual  framework  of  wargaming.  This  framework  served  as  the  basis  for  the  selection, 
design,  and  development  of  the  wargaming  assessments  explored  in  the  present  research.  The 
wargaming  process  is  described  briefly  and  a  proposed  set  of  knowledge,  skills,  and  other 
attributes  that  comprise  effective  wargaming  is  featured.  The  task  analysis  revealed  that 
essentially  the  same  general  competencies  underlie  both  co-located  and  distributed  wargaming, 
although  technical  skills  become  more  important  when  communications  are  digitally  mediated. 
Reference  to  wargaming  throughout  this  report  therefore  refers  both  to  co-located  and  distributed 
wargaming. 

The  Wargaming  Process 

Wargaming  is  a  key  component  of  Army  mission  planning.  According  to  Army  doctrine, 
it  is  the  analysis  of  multiple  courses  of  action  in  order  to  collect  information  on  their  feasibility 
and  likelihood  of  success.  After  wargaming,  the  courses  of  action  analyzed  are  compared  and 
the  superior  course  of  action  is  chosen  as  the  mission  plan.  The  following  description  of  the 
wargaming  process  is  focused  on  wargaming  as  applied  to  planning  in  a  time-constrained 
environment  because  interviews  with  SMEs  indicated  that  this  is  the  environment  in  which 
wargaming  is  commonly  done.  In  a  time-constrained  environment,  only  a  single  course  of  action 
is  analyzed. 

The  purpose  of  wargaming  a  single  course  of  action  is  to  synchronize  the  functional  areas 
on  the  battlefield,  or  battlefield  operating  systems  (BOS),  for  accomplishing  the  mission  and  to 
identify  any  weaknesses  in  the  course  of  action  that  must  be  addressed.  It  is  an  interactive 
information-gathering  exercise  through  which  the  staff  and  commander  calibrate  their  image  of 
the  battlefield  (a  dynamic  mental  model  of  the  battlefield  and  its  contextual  surroundings;  Kahan, 
Worley,  &  Stasz,  1989)  and  determine  the  means  for  allocating  resources  to  accomplish  the 
course  of  action  and  its  associated  branches/sequels.  The  overarching  goal  of  wargaming  a 
single  course  of  action  is  to  develop  a  shared  image  of  the  battlefield  among  the  commander  and 
staff  to  produce  detailed  orders  and  to  allow  maximum  flexibility  in  responding  to  enemy  action 
during  execution. 

Figure  1  below  depicts  the  input,  process,  and  outcomes  of  wargaming.  Wargaming  is 
situated  near  the  end  of  the  military  decision-making  process  (MDMP),  so  the  figure  represents 
wargaming  in  isolation  of  the  MDMP  steps  that  precede  it  and  follow  it.  The  present  research 
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focused  on  wargaming,  although  the  other  steps  of  the  MDMP-particularly  mission  analysis-are 
worthy  subjects  of  similar  study. 

The  cognitive  task  analysis  revealed  that  staffs  experience  difficulty  with  multiple  aspects 
of  wargaming,  which  often  precludes  their  ability  to  accomplish  an  accurate,  shared,  and  flexible 
image  of  the  battlefield.  First,  staffs  have  difficulty  considering  the  implications  of  friendly  and 
enemy  action  for  the  use/availability  of  personnel  (e.g.,  casualties),  resources  (e.g.,  medical 
supplies,  ammunition)  and  combat  support  (e.g.,  maintenance  and  fueling,  retransmission). 
Neglect  of  these  implications  results  in  a  plan  that  fails  because  it  is  not  robust  to  the  actual 
conditions  of  the  battlefield.  Second,  staffs  have  difficulty  considering  the  multiple  possible 
enemy  reactions  to  friendly  actions.  Plans  based  on  the  most  likely  or  most  deadly  enemy  course 
of  action  fail  when  the  enemy  follows  an  unexpected  course  of  action.  Third,  staffs  have 
difficulty  recognizing  decision  points  on  the  battlefield  and  their  indicators.  Neglect  of  decision 
points  reduces  the  commander’s  ability  to  react  to  mission  events  in  a  way  that  is  consistent  with 
his  overall  intent  and  the  mission  plan.  Fourth,  staffs  have  tremendous  difficulty  synchronizing 
the  BOSs  such  that  the  capability  of  each  BOS  is  leveraged  to  achieve  a  decisive  result.  Poorly 
synchronized  BOSs  fail  to  achieve  coordinated,  overwhelming  combat  power  and  increase  the 
risk  of  fratricide. 


Figure  1.  The  Process  of  Wargaming  a  Single  Course  of  Action. 

The  individual  and  collective  competencies  comprising  effective  wargaming  process  and 
outcomes  are  not  well  understood  (though  see  Olmstead,  1 992).  That  is,  the  tasks  that  should  be 
completed  for  a  war  game  to  be  considered  successful  are  understood  (Mullen  et  al.,  1997),  and 
the  difficulty  that  staffs  have  with  completing  these  tasks  is  well  recognized  (e.g.,  CALL,  1998, 
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2003),  but  it  is  less  well  known  what  the  sources  of  difficulty  are.  The  team  training  literature 
(e.g.,  Cannon-Bowers,  Tannenbaum,  Salas,  &  Volpe,  1995)  was  explored  to  gain  an 
understanding  of  the  individual  and  collective  competencies  that  determine  the  effectiveness  of 
the  wargaming  process  and  that  reflect  the  execution  of  effective  wargaming  tasks. 

Figure  2  shows  the  conceptual  framework  developed  to  characterize  the  competencies 
that  comprise  wargaming  effectiveness.  It  is  proposed  here  that  these  individual  and  collective 
competencies  differentiate  successful  from  unsuccessful  wargamers  as  defined  by  the  quality  of 
their  mission  plan.  This  framework  is  not  meant  to  present  an  exhaustive  list  of  competencies. 
Rather,  this  selection  of  competencies  reflects  the  belief  in  their  importance  relative  to  other 
competencies  and  a  focus  on  competencies  that  can  be  modified  through  instruction  or 
experience  (i.e.,  organizational  influences  on  wargaming  effectiveness  are  excluded).  The 
bolded  competencies  in  the  figure  are  those  competencies  selected  for  detailed  study  in  this 
research.  Each  of  these  competencies  is  described  below. 


Figure  2.  Determinants,  Processes,  and  Outcomes  of  Effective  Wargaming. 
Note:  Only  bolded  competencies  were  subjected  to  detailed  study  in  the  present  research. 
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Determinants  of  War  gaming  Effectiveness 

Determinants  of  wargaming  effectiveness  are  those  individual  and  team-related 
knowledge,  skills,  and  other  attributes  that  serve  as  a  foundation  for  effective  wargaming 
process.  Without  sufficient  development  of  wargaming  determinants,  wargaming  process  cannot 
be  effective.  In  addition,  determinants  of  wargaming  effectiveness  may  contribute  directly  to 
effective  wargaming  outcomes.  Some  determinants,  such  as  tacit  knowledge  for  wargaming  and 
wargaming  procedural  knowledge,  may  be  developed  through  practice  of  wargaming.  Other 
determinants,  such  as  team-related  motivation  and  analytical  skills/critical  thinking  may  be 
developed  outside  of  wargaming  experience.  Described  below  are  four  determinants  that  were 
selected  for  further  exploration  in  the  present  research. 

Critical  Thinking/Analogical  Reasoning.  The  purpose  of  the  staff  during  mission 
planning  and  execution  is  to  gather  and  process  information  that  the  unit  commander  will  use  as 
the  basis  for  his  decision-making.  The  critical  analysis  of  incoming  information  is  challenging 
for  staff  officers  (White,  2001).  With  the  introduction  of  digital  command  and  control  systems, 
it  is  now  more  difficult  and  more  important  that  they  effectively  analyze  the  volumes  of 
incoming  data  and  identify  the  specific  implications  of  information  for  mission  success  (Langley, 
2004).  The  ability  to  think  critically/reason  analytically  enables  staff  officers  to  sift  through 
incoming  data  with  the  commander’s  information  requirements  in  mind,  select  relevant  data  for 
further  analysis,  and  communicate  information  to  the  commander  in  a  directed,  timely  manner. 
During  wargaming,  staff  officers  must  think  critically/reason  analytically  in  order  to 
communicate  efficiently  and  effectively  with  one  another  and  to  keep  wargaming  activity 
focused  on  the  commander’s  intent. 

Knowledge  of  Own  Roles  and  Roles  of  Others.  During  wargaming,  the  workflow  of  the 
staff  is  reciprocally  interdependent  (Tesluk,  Mathieu,  Zaccaro  &  Marks,  1997).  That  is,  the  staff 
officers  involved  each  represent  a  different  functional  area  (e.g.,  intelligence,  air  defense,  etc.) 
and  must  work  closely  with  one  another  to  produce  a  refined  mission  plan  that  maximally 
leverages  the  capabilities  in  their  area  of  interest.  Staff  officers  contribute  to  refining  the  plan  by 
reporting  the  implications  of  the  current  situation  in  their  area  of  interest  for  how  mission  events 
will  play  out.  Knowledge  of  one’s  own  role  and  the  roles  of  others  on  the  staff  enables  staff 
officers  to  understand  each  other’s  information  needs  and  to  effectively  meet  those  needs 
(Cannon-Bowers  et  al.,  1995;  Olmstead,  1992). 

Tacit  Knowledge  for  Wargaming.  The  procedure  for  conducting  a  wargame,  and  its 
desired  outcome,  is  explicitly  described  in  doctrine  (FM  5-0;  DA,  2002).  However,  conducting 
an  effective  wargame  requires  an  understanding  of  the  process  that  goes  beyond  knowing 
explicit  procedure.  Staff  officers  must  understand  the  purpose  of  wargaming  in  order  to  engage 
in  many  of  the  tasks  that  are  required  for  an  effective  war  game.  Specifically,  they  must 
understand  that  producing  doctrinal  wargaming  products,  such  as  the  synchronization  matrix,  is 
not  the  goal  of  wargaming  but  a  record  of  its  results  and  that  the  quality  of  the  results  is 
dependent  upon  an  integrated  staff  effort. 

Tacit,  or  experience-based  knowledge  for  wargaming  provides  staff  officers  with  a  more 
nuanced  understanding  of  the  wargaming  process  than  is  explicitly  stated  in  doctrine,  and 
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enables  them  to  translate  this  understanding  into  action  through  more  in-depth  analysis  during 
the  war  game.  For  example,  a  less  experienced  staff  may  estimate  combat  losses  without  further 
analysis  because  the  estimate  is  enough  to  fill  out  a  cell  in  the  synchronization  matrix.  A  more 
experienced  staff  would  analyze  what  implications  combat  losses  have  for  the  commander’s 
decision-making. 

Team-Related  Motivation.  The  ability  of  the  staff  to  wargame  effectively— to  form  a 
cohesive  team  and  synchronize  their  efforts-is  in  part  a  function  of  the  motivation  of  the 
individual  team  members  to  collaborate  with  one  another  and  to  create  a  team  product  of  which 
everyone  can  be  proud.  Individual  motivation  to  invest  resources  into  team  cohesion  and 
synchronization  stems  from  perceptions  of  the  utility  of  performing  well  as  a  team,  the  utility  of 
investing  effort  in  improving  team  performance,  and  the  relation  between  one’s  own  effort 
allocation  and  team  performance  (Kanfer  &  Ackerman,  1989). 

Motivation  becomes  an  important  determinant  of  performance  when  the  application  of 
additional  effort  to  a  task  can  produce  performance  gains  (i.e.,  performance  is  resource-limited  - 
it  is  not  at  ceiling  or  constrained  by  external  forces;  Kanfer  &  Ackerman,  1989).  Wargaming  is 
one  task  where  performance  is  generally  not  at  ceiling  and  motivation  is  required  to  work  around 
the  numerous  external  pressures  against  effective  performance  (e.g.,  turnover,  command  climate, 
battlefield  chaos,  etc.). 


Effective  Wargaming  Processes 

Effective  wargaming  processes  are  enabled  when  staff  officers  possess  critical/analytical 
skills,  recognize  each  other’s  information  needs,  understand  the  purpose  of  wargaming  and  are 
motivated  to  work  together  on  a  group  product.  Staff  officers  prepared  in  this  manner  will 
communicate  better  with  one  another  and  think  more  adaptively  during  the  wargaming  process. 
These  wargaming  processes  are  described  in  more  detail  below. 

Team  Communication 

Because  wargaming  involves  a  reciprocally  interdependent  workflow  (Tesluk  et  al., 
1997),  effective  information  sharing  among  staff  officers  is  critical  for  producing  a  refined 
course  of  action.  For  each  task  that  must  be  completed  during  a  war  game,  staff  officers  must 
share  specific  information  about  their  area  of  interest  such  that  all  functional  areas  are  integrated 
into  the  refined  plan.  Effective  information  sharing  eludes  many  staffs,  with  some  staff  officers 
(e.g.,  the  Logistics  Officer  [S4])  routinely  left  out  of  the  wargaming  process. 

Adaptivity  of  Team  Thought 

Automatized  adaptive-thinking  skills  have  been  recognized  as  a  critical  aspect  of  expert 
command  decision-making  (Lussier,  Shadrick  &  Prevou,  2003).  Expert  commanders  are  tuned 
into  their  environment  and  recognize  the  implications  of  particular  environmental  conditions  for 
making  rapid  decisions  in  response  to  unforeseen  events.  Staff  officers  must  anticipate 
unforeseen  events  during  mission  execution  by  evaluating  during  wargaming  the  same  factors 
that  the  commander  must  consider  when  he  makes  decisions.  In  this  way,  wargaming  supports 
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the  commander’s  decision  making  by  ensuring  that  the  manpower  and  resources  can  be  available 
when  decisions  must  be  made.  Adaptivity  of  team  thought  among  staff  officers  enables  them  to 
consider  “What  if?”  on  the  basis  of  conditions  present  in  the  battle  situation  and  to  make 
contingency  plans  to  address  possible  events  during  execution.  This  creates  a  flexible  plan  that 
is  responsive  to  a  range  of  mission  events. 

Effective  Wargaming  Outcomes 

As  stated  previously,  the  desired  outcome  of  wargaming  is  a  refined  plan  and  a  shared 
visualization  of  the  intended  flow  of  battle  and  the  triggers  for  the  execution  of  contingency 
plans.  This  outcome  stems  directly  from  effective  communication  and  adaptive  thinking  among 
the  staff  officers  and  indirectly  from  individual  and  team-related  knowledge,  skills,  and  attitudes. 
Evaluation  of  the  quality  of  a  particular  wargaming  outcome  therefore  involves  assessing  shared 
battle  visualization  and  the  degree  to  which  the  mission  plan  has  integrated  the  capabilities  of 
each  staff  officer’s  functional  area. 

Shared  Battlefield  Visualization 

Shared  battlefield  visualization  can  be  characterized  as  shared  situation  awareness  ([SA], 
e.g.,  Endsley  &  Smolensky,  1998).  It  is  (a)  awareness  of  the  mission  plan  elements  and  their 
locations  (Level  1  SA);  (b)  understanding  of  how  plan  elements  are  synchronized  in  time  and 
space  and  the  implications  of  their  success/failure  for  one  another  (Level  2  SA);  and  (c) 
projection  of  how  these  elements  will  function  as  the  plan  is  executed  and  where  key  decisions 
will  need  to  be  made  (Level  3  SA).  Shared  SA  is  a  critical  outcome  of  wargaming  because  it  is 
the  means  by  which  the  command  and  control  team  anticipate  each  other’s  actions  during 
mission  execution. 

Integrated  Mission  Plan 

The  integrated  mission  plan  maximally  leverages  the  capabilities  of  each  staff  officer’s 
functional  area  by  synchronizing  the  efforts  of  each  area  in  time  and  space.  The  integrated 
mission  plan  is  a  critical  outcome  of  wargaming  because  its  development  requires  careful 
consideration  of  the  status  and  constraints  of  each  functional  area  and  the  implications  of  these 
factors  for  how  mission  events  can  play  out.  This  consideration  enhances  shared  SA  and  reduces 
the  risk  of  fratricide  resulting  from  uncoordinated  mission  events. 

Assessments  of  Wargaming  Effectiveness 

This  section  presents  the  assessments  that  were  designed  to  capture  all  three  aspects  of 
wargaming  performance  (determinants,  processes,  outcomes).  All  assessments  were  designed  in 
close  collaboration  with  SMEs  who  provided  input  on  the  nature  of  the  “test  items”  required  to 
capture  the  psychological  constructs  of  interest,  shared  insights  regarding  scoring  rubrics  and 
algorithms,  and  offered  their  opinions  regarding  the  ability  of  the  completed  assessments  to 
capture  the  construct  they  were  intended  to  assess.  All  of  the  assessments  were  designed  for 
administration  either  via  computer  or  via  paper-and-pencil  administration. 
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Critical  Thinking/Analogical  Reasoning 

To  assess  staff  officers’  ability  to  identify  and  communicate  relevant  information,  an 
assessment  was  designed  based  on  a  classroom  exercise  developed  by  an  instructor  at  the  U.S. 
Army  Command  and  General  Staff  College  (CGSC).  The  exercise  is  used  as  an  introduction  to 
the  Critical  Reasoning/Creative  Thinking  course  taught  at  CGSC.  It  is  intended  to  stimulate 
students’  understanding  of  their  own  ability  to  think  critically  and  creatively  under  pressure.  In 
this  exercise,  students  are  asked  to  review  a  set  of  36  PowerPoint  slides  comprising  a 
hypothetical  mission  analysis  briefing.  They  are  told  that  the  briefing  is  too  long  and  that  they 
have  5  minutes  to  reduce  the  slides  to  10  in  order  to  communicate  the  most  important 
information  in  the  briefing. 

This  instructional  exercise  was  adapted  for  assessment  [and  called  the  Mission  Analysis 
Briefing  Exercise  (MABE)]  by  developing  criteria  for  scoring  students’  abbreviated  briefings. 
The  score  for  this  exercise  is  the  number  of  slides  in  the  abbreviated  briefing  containing  relevant 
information  divided  by  the  total  (reduced)  number  of  slides  (possible  range  =  0-1).  Slides 
containing  relevant  information  were  determined  in  collaboration  with  the  instructor  who 
designed  the  exercise. 

In  the  computer-based  version  of  the  MABE,  students  are  told  they  must  shorten  the 
briefing  to  15  slides  and  are  given  15  minutes  to  complete  the  exercise.  The  computer-based 
version  of  the  MABE  was  made  slightly  easier  so  that  (a)  individual  differences  in  examinee 
familiarity  with  PowerPoint  did  not  create  differences  in  scores  on  the  exercise;  and  (b)  the 
exercise  would  not  be  so  challenging  for  examinees  that  they  disengaged  from  completing  it. 

The  CGSC  instructor  who  created  the  exercise  stated  that  students  initially  balk  at  the 
challenging  exercise  instructions  and  must  be  encouraged  to  continue.  Because  an  instructor  is 
not  involved  in  the  computer-based  administration  of  the  MABE,  simplifying  the  exercise 
seemed  a  satisfactory  alternative. 

Knowledge  of  Own  Roles  and  Roles  of  Others 

The  Staff  Roles  Knowledge  Assessment  was  designed  to  capture  examinee  understanding 
of  the  information  needs  of  core  CABTF  staff  officers.  The  Staff  Roles  Knowledge  Assessment 
has  42  items,  each  of  which  presents  a  key  wargaming  task  that  must  be  accomplished  for  the 
war  game  to  be  successful  (see  Mullen  et  al.,  1997).  Examinees  are  asked  to  indicate  which  of 
nine  staff  officers  must  share  information  in  order  to  perform  the  task  effectively.  Some  example 
items  are  shown  below.  The  complete  assessment  is  provided  in  Appendix  B  with  correct 
answers  indicated. 

Determine  the  command  and  control  requirements  for  minimizing  the  effect  of 

destruction  of  friendly  command  posts  (CPs). 

□  XO  □  SI  □  S2  □  S3  □  S4  □  SIGO  □  FSO  □  ADO  □  ENG 
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Determine  optimal  employment  of  intelligence  collection  assets. 

□  XO  □  SI  □  S2  □  S3  □  S4  □  SIGO  □  FSO  oADO  dENG 

Identify  triggers  for  the  initiation  of  direct  and  indirect  fires. 

□  XO  □  SI  □  S2  □  S3  □  S4  □  SIGO  □  FSO  □  ADO  □  ENG 

For  each  item,  the  score  is  the  number  of  staff  officers  correctly  identified  minus  the 
number  of  staff  officers  incorrectly  identified  (i.e.,  “false  alarms”).  The  total  score  is  the  average 
of  the  2-scores  for  each  of  the  42  items  (2-scores  were  used  to  ensure  that  all  items  comprising 
the  total  score  were  on  the  same  scale).  Answers  to  each  item  were  determined  by  review  of 
doctrine  and  SME  input.  The  SME  who  provided  input  was  among  those  who  identified  the  key 
wargaming  tasks  on  which  the  assessment  is  based. 

Tacit  Knowledge  for  Wargaming 

To  capture  the  level  of  development  of  examinees’  understanding  of  the  purpose  of 
wargaming  a  short,  five-question  multiple-choice  quiz  was  designed.  Each  question  asks  about 
the  purpose  of  some  aspect  of  wargaming.  For  example,  one  question  asks: 

The  wargaming  process  is  conducted  using  multiple  iterations  of  an  action- 

reaction-counteraction  (ARC)  cycle.  Why? 

a.  The  iterative  ARC  cycle  involves  a  logical  sequence  of  action. 

b.  The  iterative  ARC  cycle  reveals  the  impact  of  timing  on  friendly  and  enemy  action. 

c.  The  iterative  ARC  cycle  supports  wargaming  both  offensive  and  defensive  battles. 

d.  The  iterative  ARC  cycle  simulates  how  the  mission  will  play  out  if  the  enemy  follows 
a  particular  (e.g.,  most  probable)  COA. 

To  answer  questions  correctly,  examinees  must  have  an  understanding  of  the  purpose  of 
wargaming  that  goes  beyond  what  is  explicitly  stated  in  doctrine.  Correct  answers  and 
distracters  were  determined  by  SME  input.  Distracters  were  designed  to  represent  either 
common  misconception  regarding  the  purpose  of  wargaming  or  generic  but  not  entirely  relevant 
statements  that  can  be  found  in  doctrine.  The  score  for  this  assessment  is  the  number  correct 
minus  .25  times  the  number  incorrect  (possible  range  =  -1.25  -  5.00).  The  entire  Wargaming 
Tacit  Knowledge  assessment  is  shown  in  Appendix  C  with  correct  answers  indicated. 

Team-Related  Motivation 

To  assess  team-related  motivation,  a  12-item  survey  was  designed  in  which  examinees 
are  asked  to  rate  the  truth  of  each  item  as  Definitely  True,  Largely  True,  Depends,  Largely  False, 
Definitely  False.  Eight  items  were  designed  to  capture  examinees’  perceptions  of  the  utility  of 
staff  performance  (e.g.,  “Staff  positions  provide  experiences  that  are  important  for  developing 
command  skills.”).  One  item  was  designed  to  capture  examinees’  perceptions  of  the  utility  of 
investing  effort  in  improving  team  performance  (“Staff  performance  can  only  be  so  good; 
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External  factors  (e.g.,  rapid  staff  turnover)  exert  a  strong  limiting  influence.”)-  Three  items  were 
designed  to  capture  examinees’  perceptions  of  the  relation  between  individual  investment  of 
effort  and  the  improvement  of  team  performance  (e.g.,  “Improved  team  cohesion  is  worth  the 
effort  involved  in  developing  it.”).  The  complete  listing  of  the  items  in  the  Team-Motivation 
Survey  is  shown  in  Appendix  D. 

Team  Communication 

To  assess  team  information  sharing,  two  wargaming  observation  checklists  were 
designed  based  roughly  on  the  targeted  acceptable  responses  to  generated  events  or  tasks 
(TARGETS)  method  (Fowlkes,  Lane,  Salas,  Franz  &  Oser,  1994).  Each  checklist  features  a 
subset  of  tasks  from  the  Staff  Roles  Knowledge  Assessment,  selected  based  on  the  likelihood  of 
their  occurring  during  the  ongoing  classroom  wargaming  that  would  be  observed  as  part  of  the 
present  research.  Because  two  different  wargaming  mission  scenarios  were  to  be  observed,  the 
tasks  in  each  checklist  differ  slightly  depending  on  the  nature  of  the  scenario  (offensive 
maneuver  versus  cavalry  operations).  Doctrine  and  SME  review  were  used  to  identify  the 
information  that  must  be  shared  in  order  for  each  of  the  selected  tasks  to  be  completed 
effectively.  An  example  wargaming  task  and  its  associated  information-sharing  requirements  is 
shown  below  in  Table  1  (for  the  complete  checklists,  see  Appendix  E). 

Table  1 

Example  Item  from  a  Team  Communication  Checklist 


Task 

Information  Shared 

Rating 

Comments 

Determine  High 
Priority  Targets 

Templated  location  of  the  enemy 
and  key  enemy  assets  (e.g., 
artillery,  C2  nodes) 

Concept  of  maneuver 

Re-supply  rates  for  select 
munitions 

A  key  difference  between  the  present  checklists  and  a  TARGETS  checklist  is  that  the 
occurrence  of  tasks  to  be  observed  using  the  present  checklists  is  dependent  on  the  team’s 
effectiveness,  rather  than  on  the  design  of  the  team-performance  situation  (i.e.,  more  effective 
teams  would  attempt  more  tasks).  That  is,  in  the  TARGETS  method,  team-performance 
scenarios  are  carefully  designed  to  present  tasks  that  require  team  communication  (e.g.,  an 
equipment  malfunction).  Raters  then  indicate  whether  the  desired  communication  occurred. 
Because  validation  of  the  checklists  involved  observing  ongoing  classroom  wargaming,  special 
purpose  scenarios  that  would  specifically  elicit  certain  communication  behaviors  could  not  be 
used.  Carefully  selecting  tasks  that  were  likely  to  occur  during  student  wargaming  (routine 
targets)  seemed  a  suitable  alternative.  This  design  allows  capture  of  the  completion  of 
wargaming  tasks  independently  from  the  effectiveness  of  team  communication  and  provides  a 
means  for  focusing  observer  attention— the  primary  advantage  of  using  the  TARGETS  method. 
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A  second  difference  is  that  the  Team  Communication  checklists  require  a  greater  degree 
of  observer  judgment  because  ratings  of  information  sharing  were  not  binary  (i.e.,  information 
shared/not  shared).  Raters  use  0,  .5  or  1  to  indicate  the  degree  to  which  information  was  shared. 
Similarly,  each  task  is  rated  0,  .5  or  1  for  its  level  of  completion.  A  non-binary  rating  scale  was 
used  because  teams  can  address  a  wargaming  task  (e.g.,  integrate  maneuver  with  fire  support) 
without  necessarily  completing  it.  In  addition,  teams  could  share  some  of  the  required 
information,  but  not  all  of  it. 

The  Team  Communications  Checklists  have  three  scores:  a  task  score,  an  information¬ 
sharing  score,  and  a  total  score.  The  task  score  is  the  average  task  rating  for  all  tasks  in  the 
checklist  (possible  range  =  0-1).  The  information-sharing  score  is  the  average  sum  of  the 
average  item  ratings  within  each  of  task  (possible  range  =  0-1).  The  total  score  is  the  sum  of  the 
task  score  and  the  information-sharing  score  divided  by  two  (possible  range  =  0-1),  representing 
a  combined  assessment  of  the  accomplishment  of  wargaming  tasks  and  team  communication. 

Adaptivity  of  Team  Thought 

To  assess  examinees’  adaptive-thinking  skills  during  the  war  game,  two  interactive 
observer  checklists  were  created  based  on  the  Think  Like  a  Commander  (TLAC)  training  method 
(Lussier  et  al.,  2003).  Using  the  mission  scenarios  to  be  wargamed  in  the  course  sessions  that 
would  be  observed,  “What  if?”  questions  about  the  mission  were  identified.  Because  two 
different  wargaming  mission  scenarios  were  to  be  observed,  the  “What  if?”  questions  in  each 
checklist  differ  slightly  depending  on  the  nature  of  the  scenario  (obstacle  breaching  versus 
cavalry  operations).  Observers  using  a  TLAC  Checklist  ask  students  these  “What  if?”  questions 
during  the  war  game.  As  examinees  answer  a  question  posed  by  the  observer,  the  observer  rates 
the  quality  of  the  examinee  response  on  a  scale  of  0-2.  Anchors  for  the  rating  scale  were 
determined  by  SME  input,  with  example  “0-”  and  “2-quality”  answers  for  each  question 
provided  to  the  observer.  The  score  for  the  TLAC  Checklist  is  the  average  of  the  question  scores 
earned  by  the  students  (possible  range  =  0-2).  An  example  question  and  its  rating  scale  anchors 
are  shown  below  in  Table  2  (for  the  complete  TLAC  Checklists,  see  Appendix  F). 

Table  2 

Example  Think  Like  a  Commander  Checklist  Item 


HESllIllSMi 

Example  “0  Quality” 

Example  “2  Quality” 

What  cover  and 
concealment  can 
the  enemy  take 
advantage  of  in 
order  to  deceive 
1-22  CAV 
regarding  the 
size/strength  of 
threat  forces? 

Solutions  that  do  not  include 
consideration  of  local  populace 

Use  the  civilian  population  to  blend  into  the 
terrain 

Solutions  that  do  not  consider  the 
effects  of  urban  terrain 

Maximize  the  use  of  forces  within  urban 
areas  where  intelligence  collection  and 
communication  capabilities  are  weakened  for 
U.S.  forces 

Solutions  that  are  not  based  on 
fighting  an  asymmetric  enemy 
(e.g.,  students  think  in  terms  of 
large  units,  heavy  equipment,  etc.) 

Disperse  widely,  remain  in  small  teams,  and 
move  frequently  such  that  effective  estimates 
require  an  integrated  intelligence  effort 
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Shared  Battlefield  Visualization 

To  assess  shared  SA  as  a  wargaming  outcome,  an  exercise  based  on  an  Army  doctrinal 
wargaming  product  called  a  decision  support  matrix  (DSM)  was  devised.  The  DSM  requires 
staffs  to  consider  where  on  the  battlefield  the  commander  will  have  to  make  a  decision  during 
mission  execution,  what  information  he  needs  to  make  those  decisions,  and  the  options  available 
to  the  commander  at  the  time  of  decision-making.  As  shown  in  Table  3,  the  SA  Exercise 
requires  examinees  to  answer  questions  associated  with  elements  of  the  DSM,  which  correspond 
to  each  level  of  SA. 

Table  3 


Example  Situational  Awareness  Exercise  Questions 


SA  Level 

Question 

1 

What  is  the  center  of  mass  for  Named  Area  of  Interest  (NAI)  3? 

2 

If  the  enemy  strongpoint  is  located  forward  on  high  ground  at  NAI  3,  what  would 
this  reveal  about  what  the  enemy  intends  to  do? 

3 

What  should  Task  Force  (TF)  1-93  do  if  the  eastern  enemy  strongpoint  is  located 
forward  on  high  ground? 

Only  one  of  the  wargaming  mission  scenarios  used  in  the  courses  to  be  observed  was 
selected  as  the  basis  for  the  SA  exercise  questions.  With  the  help  of  subject  matter  experts,  an 
exercise  “key”  was  developed  to  score  student  wargaming  outcomes.  The  entire  set  of  questions 
comprising  the  exercise  and  the  scoring  key  are  shown  in  Appendix  G.  The  individual  score  for 
this  exercise  is  the  sum  of  correct  answers,  weighted  according  to  the  level  of  SA  required  to 
answer  the  question  correctly  (possible  range  =  0-30).  The  team  score  for  this  exercise  is  the 
percent  agreement  on  correct  answers  for  each  question  (possible  range  =  0-15). 

Integrated  Mission  Plan 

To  assess  the  level  of  integration  of  the  mission  plan,  an  Integrated  Overlay  Exercise  was 
designed  based  on  one  of  the  wargaming  mission  scenarios  used  in  the  courses  to  be  observed. 

In  this  exercise  examinees  are  asked  after  they  complete  the  war  game  to  create  a  graphical 
overlay  of  each  phase  of  the  refined  mission  plan.  The  overlay  is  to  contain  the  key  elements  of 
friendly  activity  not  represented  in  the  course  of  action  sketch  used  as  input  for  the  war  game 
(i.e.,  the  activity  determined  during  wargaming),  including  logistical  assets  and  control  measures. 
Upon  completing  the  overlay,  examinees  are  asked  to  brief  the  overlay  to  an  observer,  providing 
a  rationale  for  the  placement  of  each  element. 

Examinees  earn  one  point  for  each  element  they  include  in  the  overlay  (max  =  21), 
compared  to  a  “key”  created  by  an  SME.  The  observer  rates  examinee  rationale  for  each 
element  using  a  checklist  in  which  the  components  of  rationale  for  each  element  are  listed.  An 
example  element  and  its  rationale  components  are  shown  in  Table  4.  The  score  for  each  element 
is  the  proportion  of  the  total  number  of  rationale  components  provided  by  the  student  (possible 
range  =  0-1).  The  score  for  the  exercise  is  the  number  of  elements  included  in  the  overlay 
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divided  by  the  sum  of  the  element  rationale  scores  (possible  range  =  0-1).  The  complete  list  of 
elements  and  rationale  components  is  provided  in  Appendix  H. 

Table  4 


Example  Integrated  Overlay  Checklist  Item  and  Rationale 


Element 

Rationale  Component 

Smoke 

Smoke  at  this  location  will  mask  TF  1-93  maneuver  and  protect  movement  to 
attack  positions 

Smoke  location  is  coordinated  with  S2  projections  for  weather/wind  direction 

Evaluation  of  Selected  Wargaming  Effectiveness  Assessments 

The  above-described  assessments  of  wargaming  effectiveness  can  be  considered  valid  if 
they  reliably  capture  the  constructs  they  are  intended  to  capture.  That  is,  the  assessments  must 
(a)  demonstrate,  where  applicable,  psychometric  soundness;  and  (b)  differentiate  between  more 
and  less  capable  wargamers  as  identified  by  independent  criteria.  This  section  presents  the 
approach  used  in  the  present  research  for  exploring  the  validity  of  the  assessments.  Findings  are 
also  presented  and  discussed. 

Method 

Participants.  Participants  were  49  students  in  the  AC3DL,  taught  through  the  U.S.  Army 
Armor  School  at  Fort  Knox,  KY.  These  students  were  first  lieutenants  (LT)  (approximately1 
37%),  captains  (CPT)  (approximately  55%),  and  one  major  (MAJ)  in  the  U.S.  Army  National 
Guard.  In  addition,  two  AC3DL  students  were  civilians  with  prior  Army  experience  who  were 
taking  the  course  as  part  of  a  job  requirement.  Slightly  older  than  their  counterparts  in  the 
regular  Army,  the  AC3DL  students  were,  on  average,  approximately  35  years  old.  Ninety-seven 
percent  (37/38  -  see  footnote  #1)  of  them  were  male.  Approximately  75%  of  the  National  Guard 
officers  in  the  course  were  armor  officers,  the  rest  were  air  defense  artillerymen,  engineers, 
infantrymen,  military  intelligence  officers,  and  signal  officers.  Participation  in  the  research 
occurred  as  part  of  the  students’  ongoing  coursework. 

Materials.  Of  the  wargaming  effectiveness  assessments  developed,  all  but  the  Integrated 
Overlay  Exercise  were  administered  to  participants.  The  technical  development  required  to 
implement  this  exercise  was  beyond  the  scope  of  the  present  research.  A  short  demographic 
survey  was  also  administered  (see  Appendix  I).  All  materials  that  were  not  observer  checklists 
(i.e.,  the  demographic  survey,  MABE,  Staff  Roles  Knowledge  Assessment,  Wargaming  Tacit 
Knowledge  Assessment,  Team-Related  Motivation,  SA  Exercise)  were  administered  via  the 
computer  either  as  a  Microsoft  Word  or  PowerPoint  file  sent  via  email  or  as  a  web-based  form. 


1  Eleven  students  did  not  fill  out  a  demographic  survey  so  overall  participant  data  are  approximate. 
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Procedure.  All  materials  were  administered  as  part  of  the  students’  ongoing  coursework 
in  the  AC3DL.  The  purpose  of  AC3DL  is  to  teach  junior  National  Guard  officers  the 
fundamentals  of  the  MDMP  and  basic  company  command  skills.  As  shown  in  Table  5  below, 
the  AC3DL  is  taught  in  multiple  phases  (see  Sanders,  2002  for  a  complete  description  of  this 
course). 

Table  5 

Overview  of  the  Armor  Captain’s  Career  Course  (Distance  Learning) 


Phase  I  -  Distributed 

Phase  II -Co-located 

Distributed/Seif-Study 

Distributed/Collaborative 

24  Internet-based  modules 
covering  brigade  and  battalion 
staff  skills  and  company  team 
operations 

7  sessions  to  practice  the  steps 
of  the  MDMP  as  a  battalion 
staff  using  instructor¬ 
generated  materials  (e.g., 
brigade  operations  order, 
commander’s  intent,  etc.) 

Two-weeks  practice  using  the 
MDMP  in  the  context  of  a 
fast-paced  plan-prepare- 
execute  cycle,  using  different 
instructor-generated  materials 

The  present  research  involved  assessment  of  student  wargaming  during  Phase  I  - 
Distributed/Collaborative  and  Phase  II  of  the  course.  Phase  I  -  Distributed/Collaborative  is 
comprised  of  seven  sessions  conducted  in  the  VTOC.  The  VTOC  allows  geographically 
distributed  students  to  work  collaboratively  using  text  chat,  voiceover  Internet  protocol,  and 
document  sharing.  During  the  seven  distributed/collaborative  sessions,  students  are  each 
assigned  a  CABTF  staff  duty  position,  and  practice  developing  a  mission  plan  and  troop  leading 
procedures  using  the  MDMP.  Mission  plans  generated  by  students  are  based  on  instructor¬ 
generated  materials  that  include  a  brigade  operations  order  and  commander’s  intent,  graphical 
overlays,  etc.  Students  who  pass  Phase  I  of  the  course  continue  on  to  Phase  II,  spending  two 
weeks  at  Fort  Knox  to  complete  their  training  as  a  co-located  group.  Phase  II  involves  further 
practice  using  the  MDMP  in  the  context  of  a  fast-paced  plan-prepare-execute  cycle  of  activity. 

Students  conduct  wargaming  during  the  fourth  session  of  the  distributed/collaborative 
phase  of  training  and  during  the  co-located  resident  phase.  Each  of  the  two  wargaming  events 
involves  a  different  mission  scenario.  The  scenario  used  during  the  distributed/collaborative 
phase  involves  a  mechanized  infantry  task  force  (TF  1-93),  whose  mission  is  to  draw  an  enemy 
combined-arms  reserve  element  away  from  an  adjacent  friendly  task  force  as  it  attempts  to  seize 
a  key  objective.  The  scenario  used  during  the  co-located  phase  involves  a  reconnaissance, 
surveillance,  and  target  acquisition  (RSTA)  squadron  (1-22  CAV),  whose  mission  was  to 
perform  reconnaissance  to  assist  a  Stryker  brigade  combat  team  commander  in  determining 
which  of  three  courses  of  action  to  pursue.  Because  the  distributed/collaborative  and  co-located 
phases  of  instruction  involve  the  use  of  different  mission  scenarios,  the  administration  of 
scenario-based  assessments  was  determined  in  part  by  the  phase  of  instruction  students  were  in  at 
the  time  of  data  collection. 

Assessments  of  wargaming  determinants  (MABE,  Staff  Roles  Knowledge  Assessment, 
Wargaming  Tacit  Knowledge  Assessment,  Team-Related  Motivation)  and  the  demographic 
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survey  were  administered  prior  to  student  wargaming.  Assessment  of  wargaming  process 
effectiveness  (Team  Communication  Checklist,  TLAC  Checklist)  was  conducted  during  student 
wargaming.  Assessment  of  wargaming  outcome  effectiveness  (SA  Exercise)  was  administered 
after  student  wargaming  was  completed.  As  an  independent  assessment  of  wargaming 
effectiveness,  instructors  were  asked  after  wargaming  was  over  to  rate  each  group  of  students  on 
the  quality  of  their  wargaming  performance  relative  to  doctrinal  standards 
(Poor/Fair/Good/Excellent)  and  relative  to  other  student  groups  (Below  Average/Average/Above 
Average). 

Approximately  five  classes  of  the  AC3DL  are  taught  per  year  by  two  instructors.  The  49 
students  who  participated  in  the  present  research  were  members  of  six  independent  groups  of 
students  who  were  enrolled  in  AC3DL  between  November  2004  and  September  2005  (see  Table 
6  below).  Three  of  these  groups  went  through  both  the  distributed/collaborative  and  resident 
phases  of  the  AC3DL  during  the  time  data  were  collected.  The  composition  of  these  groups 
changed  slightly  between  the  distributed/collaborative  and  resident  phases  of  instruction,  with 
students  who  missed  their  assigned  resident  phase  (e.g.,  due  to  deployment)  attending  a  later 
resident  phase.  The  remaining  groups  of  students  went  through  either  the  distributed/ 
collaborative  or  the  resident  phase  of  AC3DL  during  the  time  data  were  collected. 

Table  6  below  shows  the  assessments  that  were  administered  to  each  student  group.  Due 
to  time  constraints,  none  of  the  three  student  groups  who  went  through  both  the 
distributed/collaborative  and  resident  phases  of  the  AC3DL  during  the  present  research  was 
given  the  same  assessment  twice.  Due  to  technical  and  other  constraints  some  assessments  were 
administered  only  to  some  groups.  A  complete  explanation  of  why  some  groups  did  not  receive 
particular  assessments  is  provided  in  the  next  section. 

Table  6 


Overall  Administration  of  Wargaming  Effectiveness  Assessments 


Group 

Instructor 

Learning  Environment 

Assessments  Administered 

1 

A 

VTOC  (TF  1-93) 

Not  Applicable  (N/A) 

1 

A 

Classroom  (1-22  CAV) 

Demographic  Survey,  Staff  Roles,  Wargaming  Tacit 
Knowledge,  Team  Communication  Checklist  (1-22  CAV) 

2 

B 

VTOC  (TF  1-93) 

Demographic  Survey,  MABE,  Staff  Roles,  Wargaming 
Tacit  Knowledge,  Team  Communication  Checklist  (TF  1- 
93) 

2 

A 

Classroom  (1-22  CAV) 

TLAC  Checklist  (1-22  CAV) 

3 

A 

Classroom  (1-22  CAV) 

Demographic  Survey,  Staff  Roles,  TLAC  Checklist  (1-22 
CAV) 

4 

A 

VTOC  (TF  1-93) 

Demographic  Survey,  MABE,  Staff  Roles,  Wargaming 
Tacit  Knowledge,  Team  Communication  Checklist  (TF  1- 
93),  SA  Exercise 

4 

A 

Classroom  (1-22  CAV) 

TLAC  Checklist  (1-22  CAV) 

5 

B 

VTOC  (TF  1-93) 

Demographic  Survey,  MABE,  Staff  Roles,  Wargaming 
Tacit  Knowledge,  Team  Communication  Checklist  (TF  1- 
93),  SA  Exercise 

6 

A 

VTOC  (TF  1-93) 

Demographic  Survey,  Staff  Roles,  Wargaming  Tacit 
Knowledge,  Team  Communication  Checklist  (TF  1-93), 

SA  Exercise 
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Findings 


Assessment  Properties.  This  section  summarizes,  where  applicable,  the  psychometric 
and  other  properties  of  each  assessment  administered  as  part  of  the  present  research,  including 
the  feasibility  of  its  administration. 

Mission  Analysis  Briefing  Exercise  (MABE) 

The  MABE  was  administered  only  to  distributed/collaborative  phase  student  groups  in 
order  to  minimize  the  course  interruption  caused  by  research  data  collection.  Administration  of 
the  MABE  in  the  classroom  setting  would  have  required  the  students  to  do  the  exercise  as  a 
group,  proctored  and  timed  by  the  instructor  or  an  experimenter.  This  was  determined  to  be  too 
disruptive  to  the  flow  of  classroom  activities  and  likely  to  reduce  student  motivation  to  perform 
well  during  the  exercise.  Web  administration  of  the  MABE  using  the  VTOC  allowed  students  to 
do  the  exercise  individually  with  the  instructions  and  timing  accomplished  by  the  computer. 
Group  1  was  not  administered  the  MABE  or  any  other  assessments  due  to  the  high  level  of 
difficulty  this  group  of  students  had  with  the  course  materials. 

The  MABE  therefore  was  administered  to  three  student  groups  totaling  16  participants. 
However,  only  five  students  completed  the  MABE.  The  MABE  proved  difficult  to  administer 
due  to  limitations  in  the  Internet  speed  available  to  most  students  in  the  distributed/collaborative 
phase  of  instruction.  That  is,  to  complete  the  exercise  using  the  VTOC,  students  had  to 
download  the  PowerPoint  briefing  onto  their  personal  computers  in  order  to  modify,  save  it,  and 
upload  it  in  the  time  allowed.  The  1 5-minute  exercise  timer  starts  when  the  download  of  the 
briefing  is  initiated  and  ends  when  the  upload  of  the  modified  briefing  is  completed.  Students 
with  slow  Internet  speeds  were  unable  to  complete  this  process  in  the  time  allowed  and  became 
frustrated  with  their  lack  of  control  in  successfully  completing  the  exercise.  For  this  reason,  the 
MABE  was  not  administered  to  Group  6,  the  final  distributed/collaborative  student  group 
observed  in  this  research. 

Among  the  five  students  who  completed  the  MABE,  scores  ranged  from  .52  to  .64,  with 
three  students  earning  a  .64.  These  findings,  albeit  limited  in  scope,  suggest  that  range 
restriction  may  be  a  problem  when  administering  the  MABE  to  larger  groups.  This  problem 
could  be  solved  in  one  of  two  ways.  First,  the  scoring  method  could  be  changed.  That  is,  rather 
than  simply  summing  the  number  of  slides  in  the  abbreviated  briefing  containing  relevant 
information  and  dividing  by  the  total  (reduced)  number  of  slides,  a  weighted  sum  of  the  slides 
containing  relevant  information  could  be  derived  and  divided  by  the  total  (reduced)  number  of 
slides.  The  weighting  scheme  could  be  determined  by  the  relative  importance  of  the  relevant- 
information  slides  present  in  the  reduced  briefing.  Such  a  change  to  the  scoring  method  could  be 
effective  because  although  students  tended  to  reduce  the  briefing  to  roughly  the  same  number  of 
relevant  slides  (8-1 1),  they  often  selected  different  slides  to  include. 

Second,  the  content  of  the  slides  in  the  unabbreviated  briefing  could  be  modified  to 
reflect  more  systematic  design  of  the  assessment  itself.  The  learning  exercise  on  which  the 
MABE  was  based  was  not  intended  as  an  assessment.  For  this  reason,  it  was  not  designed 
specifically  to  reveal  individual  differences  in  critical  thinking/analytical  reasoning,  but  simply 
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to  raise  awareness  about  what  critical  thinking/analytical  reasoning  “feels  like.”  Redesign  of  the 
MABE  would  require  identifying  the  indicators  of  different  levels  of  critical  thinking/analytical 
reasoning  as  they  relate  to  the  information  that  could  be  present  in  a  mission  analysis  briefing. 
For  example,  students  with  less  well-developed  levels  of  critical  thinking/analytical  reasoning 
might  know  to  include  slides  containing  such  relevant  information  as  risks  or  recommendations 
but  might  not  know  to  exclude  slides  that  present  weather/illumination  data  without 
interpretation. 

Staff  Roles  Knowledge  Assessment 

The  Staff  Roles  Knowledge  Assessment  was  administered  to  all  six  student  groups,  with 
37  of  49  students  completing  it.  Distributed/collaborative  phase  student  groups  took  the 
assessment  using  a  web-based  form  available  in  the  VTOC.  One  student  did  not  complete  the 
assessment  due  to  technical  difficulties  with  the  VTOC.  Co-located  student  groups  who  had  not 
already  taken  the  Staff  Roles  Knowledge  Assessment  in  the  VTOC  filled  out  a  Microsoft  Word 
file  emailed  to  them  by  their  instructor.  Eleven  students  who  were  not  present  in  a  distributed/ 
collaborative  wargaming  session  observed  during  data  collection  but  were  present  for  resident- 
phase  wargaming  did  not  complete  the  Staff  Roles  Knowledge  Assessment.  The  Staff  Roles 
Knowledge  Assessment  was  easy  to  administer  and  took  relatively  little  student  time  to 
complete. 

In  initial  pilot  testing,  the  split-half  reliability  of  the  Staff  Roles  Knowledge  Assessment 
was  quite  high  (r^  =  .94),  so  the  assessment  was  reduced  by  half  in  order  to  ease  the  workload 
on  students.  The  internal-consistency  reliability  of  the  reduced  assessment  is  also  high,  at  .85. 
The  range  in  total  scores  on  the  Staff  Roles  Knowledge  Assessment  was  -.82  to  .87,  with  a  mean 
and  standard  deviation  of  .00  and  .51,  respectively.  This  range  in  scores  suggests  that  the  Staff 
Roles  Knowledge  Assessment  is  capturing  individual  differences,  and  is  not  subject  to  a  floor  or 
ceiling  effect. 

Response-accuracy  patterns  in  the  Staff  Roles  Knowledge  Assessment  mirrored  those 
patterns  expected  on  the  basis  of  known  deficiencies  in  staff  information  sharing.  That  is,  on 
average,  students  most  accurately  identified  the  information  needs  of  the  operations  officer  and 
the  intelligence  officer,  who  are  a  central  focus  of  the  wargaming  process  (mean  percent  correct 
for  these  officers  was,  respectively,  35  and  13  percentage  points  above  the  mean  percent  correct 
for  all  officers,  which  was  35).  The  information  needs  of  other  staff  officers,  i.e.,  those 
representing  functional  areas  that  deal  with  combat  support  and  combat  service  support,  were 
poorly  understood  by  the  students  who  participated  in  the  present  research,  just  as  they  are  not 
well  understood  by  many  combat  arms  officers  in  general  (CALL,  1998).  The  mean  percent 
correct  for  these  officers  was,  on  average,  nine  percentage  points  below  the  mean  percent  correct 
for  all  staff  officers. 

An  additional,  seemingly  counterintuitive  finding  is  that  students  were  least  accurate  in 
identifying  the  information  needs  of  various  staff  officers  during  the  wargaming  of  maneuver 
and  intelligence  tasks  (mean  percent  correct  was  32  and  35,  respectively)  and  most  accurate  in 
identifying  the  information  needs  of  various  staff  officers  when  wargaming  combat  support  tasks 
(mean  percent  correct  for  fire  support,  maneuverability/survivability,  and  combat  service  support 
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tasks  was  47,  44,  and  44,  respectively).  This  pattern  of  accuracy  actually  is  expected  when  one 
assumes  that  students  were  more  likely  to  neglect  the  information  needs  of  other  staff  officers 
when  completing  wargaming  tasks  in  their  area  of  expertise.  Most  of  the  students  in  the  AC3DL 
were  armor  officers,  those  officers  who,  when  inexperienced,  are  more  likely  to  underestimate 
the  information  needs  of  staff  officers  not  directly  involved  with  maneuver.  In  addition,  the 
combat  support  wargaming  tasks  included  in  the  Staff  Roles  Knowledge  Assessment  generally 
involved  fewer  staff  officers  whose  role  in  wargaming  the  task  was  more  obvious.  This 
condition  made  it  easier  to  correctly  identify  most  or  all  of  the  staff  officers  involved  in  the  task. 

War  gaming  Tacit  Knowledge 

The  Wargaming  Tacit  Knowledge  Assessment  was  administered  to  five  of  the  six  student 
groups,  with  26  of  38  students  completing  the  assessment.  As  with  the  Staff  Roles  Knowledge 
Assessment,  distributed/collaborative-phase  student  groups  took  the  Wargaming  Tacit 
Knowledge  Assessment  using  a  Web-based  form  available  in  the  VTOC.  One  student  did  not 
complete  the  assessment  due  to  technical  difficulties  with  the  VTOC.  Co-located  student  groups 
who  had  not  already  taken  the  Staff  Roles  Knowledge  Assessment  in  the  VTOC  filled  out  a 
Microsoft  Word  file  emailed  to  them  by  their  instructor.  Eleven  students  who  were  not  present 
for  a  distributed/collaborative  wargaming  session  observed  during  data  collection  but  were 
present  for  resident-phase  wargaming  did  not  complete  the  Wargaming  Tacit  Knowledge 
Assessment.  Group  3,  which  wargamed  before  this  assessment  was  completed,  was  not 
administered  the  assessment.  This  assessment,  like  the  Staff  Roles  Knowledge  Assessment,  was 
easy  to  administer  and  took  little  student  time  to  complete. 

Scores  on  the  Wargaming  Tacit  Knowledge  Assessment  ranged  from  -1.25  to  5.00,  the 
widest  range  possible,  with  a  mean  and  standard  deviation  of  1 .63  and  1.53,  respectively.  The 
internal  consistency  reliability  of  this  assessment  was  .36.  This  estimate  is  quite  low  given  the 
quiz  is  intended  to  assess  a  single  construct.  However,  the  quiz  is  only  five  questions  long  and 
the  length  of  an  assessment  has  implications  for  its  reliability.  When  the  Spearman-Brown 
formula  is  used  to  determine  what  the  reliability  of  the  Wargaming  Tacit  Knowledge  Assessment 
would  be  if  it  was  as  long  as  the  abbreviated  Staff  Roles  Knowledge  Assessment  (21  items),  the 
reliability  is  satisfactory  at  .70. 

Team-Related  Motivation 

The  Team-Motivation  Survey  was  administered  to  all  six  student  groups,  with  32  of  49 
students  completing  it.  Distributed/collaborative  phase  student  groups  took  the  survey  using  a 
Web-based  form  available  in  the  VTOC.  One  student  did  not  complete  the  survey  due  to 
technical  difficulties  with  the  VTOC;  three  students  did  not  complete  the  survey  for  unknown 
reasons.  Co-located  student  groups  who  had  not  already  taken  the  Team-Motivation  Survey  in 
the  VTOC  filled  out  a  Microsoft  Word  file  emailed  to  them  by  their  instructor.  Eleven  students 
who  were  not  present  for  a  distributed/collaborative  wargaming  session  observed  during  data 
collection  but  were  present  for  resident-phase  wargaming  did  not  complete  the  Team-Motivation 
Survey.  Two  students  did  not  fill  out  the  survey  for  unknown  reasons.  It  is  possible  that  these 
two  students  overlooked  the  survey,  as  it  was  appended  to  the  end  of  the  demographic  survey  in 
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the  MS  Word  file  sent  by  the  instructor.  The  Team-Motivation  Survey  was  easy  to  administer 
and  took  relatively  little  student  time  to  complete. 

Table  7  below  shows  the  percent  of  responses  to  each  survey  item  that  indicate  positive 
levels  of  motivation.  That  is,  for  items  1,2,4,  and  7-10,  the  percentage  in  the  right  hand  column 
of  the  table  indicates  the  proportion  of  students  selecting  “Largely  True”  or  “Definitely  True”  as 
their  response  to  the  item.  For  the  remaining  items,  the  percentage  indicates  the  proportion  of 
students  selecting  “Largely  False”  or  “Definitely  False.”  As  can  be  seen  in  the  table,  several 
items  are  positively  endorsed  by  all  or  nearly  all  of  the  students  participating  in  the  present 
research.  One  exception  is  Item  12,  which  a  large  majority  of  students  (81%)  endorsed 
negatively.  At  a  general  level,  these  results  suggest  that  students  are  motivated  to  perform  well 
as  staff  officers  and  feel  that  staff  officers  are  important  in  aiding  the  commander.  However,  the 
students  also  appear  to  feel  some  lack  of  control  over  the  quality  of  staff  performance,  that 
efforts  devoted  to  team  cohesion  are  not  as  important  as  efforts  devoted  technical  self¬ 
development,  and  that  staff  positions  carry  less  prestige  than  command  positions. 

Table  7 

Percentage  of  Students  Positively  Endorsing  Motivation  Survey  Items 


Item 

Positive  Endorsement 

1 .  Staff  positions  provide  experiences  that  are  important  for 
developing  command  skills. 

88% 

2.  Good  performance  in  a  staff  position  is  required  for  promotion  to 
command. 

66% 

3.  Great  staff  members  don’t  make  history;  Great  commanders  do. 

34% 

4.  Superior  staff  performance  is  a  source  of  pride  for  individual  staff 
members. 

91% 

5.  Good  commanders  don’t  need  a  staff  to  aid  in  planning;  They  can 
already  visualize  the  battlefield  and  act  decisively. 

94% 

6.  Staff  performance  is  not  a  major  factor  in  determining  battle 
outcomes. 

94% 

7.  Technical  knowledge  (including  doctrine  and  digital  skills)  is 
critical  for  superior  staff  performance. 

94% 

8.  Improved  technical  knowledge  is  worth  the  effort  involved  in 
acquiring  it. 

100% 

9.  Team  cohesion  is  a  critical  characteristic  of  superior  staffs. 

100% 

10.  Improved  team  cohesion  is  worth  the  effort  involved  in 
developing  it. 

100% 

1 1 .  There  is  higher  payoff  for  investing  effort  to  acquiring  technical 
knowledge  than  for  investing  effort  in  developing  team  cohesion. 

47% 

12.  Staff  performance  can  only  be  so  good;  External  factors  (e.g., 
rapid  staff  turnover)  exert  a  strong  limiting  influence. 

19% 

To  examine  individual  differences  in  team-related  motivation,  students’  item 
endorsements  were  recoded  to  a  numerical,  0-4  scale  in  which  4  indicates  the  highest  level  of 
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motivation  and  0  the  lowest.  The  highest  level  of  motivation  could  be  a  “Definitely  True”  or 
“Definitely  False”  selection,  depending  on  the  question.  Team-related  motivation  “scores” 
derived  in  this  manner  ranged  from  28  to  45  (possible  range  is  0-48),  with  a  mean  of  36.47  and  a 
standard  deviation  of  4.02.  In  general,  it  appears  that  students  had  moderate  to  high  levels  of 
motivation  as  captured  by  the  survey,  and  that  there  was  relatively  limited  variance  among 
individuals  in  level  of  motivation. 

The  internal-consistency  reliability  of  the  Team-Motivation  Survey  was  .68.  When  the 
four  items  referring  to  the  prestige  of  staff  positions  (Items  2  and  3),  the  utility  of  investing  in 
team  cohesion  (Item  1 1),  and  the  relation  between  effort  investment  and  staff  performance  (Item 
12)  are  removed,  the  internal-consistency  reliability  is  .79.  This  increase  in  reliability  suggests 
that  although  the  survey  is  not  capturing  two  distinct  aspects  of  team-related  motivation,  it  is 
perhaps  capturing  a  distinction  that  respondents  make  between  the  importance  of  staff  function 
and  expertise  and  the  reality  of  its  development,  recognition,  and  payoff. 

Team  Communication  Checklist 

Five  student  groups  (four  distributed/collaborative  and  one  co-located)  were  observed  for 
the  quality  of  their  information  sharing  using  one  of  two  Team  Communication  Checklists.  The 
total  communication  score  (the  combined  average  of  task  scores  and  information  sharing  scores) 
can  range  from  0  to  1,  reflecting  both  the  degree  to  which  information  is  shared,  and  also  the 
level  of  wargaming  task  completion.  In  the  present  research,  group  total  scores  ranged  from  .28 
to  .53,  with  a  mean  of  .36  and  a  standard  deviation  of  .1 1 .  Task  scores,  which  reflect  only  the 
level  of  task  completion,  ranged  from  .41  to  .67  (possible  range  is  0-1),  with  a  mean  of  .50  and  a 
standard  deviation  of .  1 1 .  Information  sharing  scores,  which  reflect  the  degree  to  which  essential 
task  information  is  shared,  ranged  from  .09  to  .38  (possible  range  is  0-1),  with  a  mean  of  .22  and 
a  standard  deviation  of  .12.  These  scores  indicate  that  there  was  some  limitation  in  range  of 
student  group  performance  (3  of  5  total  scores  were  .28-.29;  3  of  5  task  scores  was  .41 -.46),  but 
that  performance  was  neither  at  floor  or  ceiling.  Performance  was  generally  low  and  information 
sharing  (i.e.,  information  sharing  scores)  tended  to  be  quite  spare,  albeit  variable. 

Where  two  raters  used  the  Team  Communication  checklist,  they  agreed  on  67%  of  the 
judgments  that  a  wargaming  task  had  been  attempted.  The  correlation  of  the  scores  they 
assigned  to  each  task  was  .54.  Similarly,  these  same  two  raters  agreed  on  77%  of  the  judgments 
that  an  attempt  to  share  a  particular  piece  of  information  occurred.  The  correlation  between  the 
scores  they  assigned  to  information  sharing  was  .69.  Differences  among  the  two  raters  appear  to 
come  from  differing  levels  of  experience  observing  AC3DL  wargaming  and  differing  levels  of 
familiarity  with  the  doctrinal  information  requirements  for  each  wargaming  task. 

Notably,  task  scores  were  substantially  higher  than  information  scores,  indicating  that 
students  were  attempting  wargaming  tasks,  but  generally  not  sharing  sufficient  information  to 
complete  the  tasks  according  to  doctrinal  and  expert  standards.  This  appears  to  have  occurred 
for  two  reasons.  First,  students,  particularly  those  in  the  distributed/collaborative  phase  of 
instruction,  tended  to  conduct  wargaming  with  the  goal  of  filling  out  the  synchronization  matrix 
used  to  record  wargaming  results,  rather  than  the  goal  of  refining  the  plan.  This  focus  on 
recording  results  as  opposed  to  refining  the  plan  appears  to  have  occurred  in  part  because  the 
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course  instructors  use  a  completed  portion  of  the  synch  matrix  to  introduce  students  to  the 
wargaming  process.  They  brief  the  initial,  completed  portion  of  the  matrix  (Phase  I  of  the 
operation),  but  do  not  model  how  that  portion  is  derived.  The  consequence  is  that  students 
perform  several  of  the  tasks  listed  in  the  Team  Communication  Checklist,  but  only  to  the  extent 
required  to  fill  in  the  synchronization  matrix.  This  can  be  accomplished  without  thinking  more 
broadly  about  refining  the  plan  and  to  a  large  extent  without  communication. 

Second,  students  do  not  have  the  technical  means  (i.e.,  through  simulation)  or  experience 
to  really  visualize  the  mission,  to  think  in  detail.  For  example,  they  will  determine  the  priority  of 
air  defense  coverage  as  “area  coverage,”  but  generally  do  not  get  more  specific  than  that  (i.e., 
what  the  basis  for  that  determination  is,  other  than  that  “area  coverage”  is  specified  in  the 
brigade  operations  order  that  is  part  of  the  mission  scenario).  Students  would  decide  air  defense 
coverage  without  ever  discussing  what  units,  specifically,  were  going  to  be  covered  and  how  to 
use  coverage  to  mass  combat  power. 

Support  for  these  ideas  comes  from  the  fact  that  co-located  student  wargaming,  in  which 
the  recording  technique  was  less  central  and  there  was  better  simulation  of  actual  operational 
conditions,  earned  a  substantially  higher  task  and  information  sharing  score  than  all  of  the 
distributed  wargaming  observed.  Yet,  even  co-located  wargaming  was  not  optimal  (task  score  = 
.67,  information  sharing  score  =  .38).  It  is  possible  that  co-location  generated  or  at  least 
contributed  to  better  performance.  However,  observation  of  student  behavior  in  both  conditions 
suggests  that  communication  differences  were  most  likely  due  to  a  greater  ability  to 
conceptualize  the  purpose  of  wargaming  through  greater  focus  on  refining  the  plan  and  better 
simulation  of  operational  conditions.  The  relative  veracity  of  these  two  explanations  remains  to 
be  tested.  In  any  case,  these  findings  suggest  that  the  Team  Communication  Checklists  are 
useful  for  understanding  more  fully  what  was  happening  during  wargaming.  In  the  present 
research,  they  were  sensitive  to  limitations  in  the  technical  and  experiential  support  that  students 
needed  to  wargame  optimally. 

The  Team  Communication  Checklists  were  equally  unobtrusive  during  distributed  and 
co-located  observation.  One  challenge  encountered  when  using  the  checklist  was  identifying 
who  was  sharing  information.  During  distributed  observation,  it  was  often  difficult  to  tell  who 
was  speaking  and,  consequently,  what  staff  role  they  were  playing  during  the  war  game.  In  both 
distributed  and  co-located  wargaming,  students  often  played  more  than  one  staff  role  or  staff 
roles  went  unfilled  due  to  lack  of  students  (e.g.,  one  group  had  only  three  students).  For  these 
reasons,  it  turned  out  to  be  unproductive  to  attempt  tracking  who  was  communicating  relative  to 
who  should  be  communicating  according  to  doctrinal  standards. 

Think  Like  a  Commander  (TLAC)  Checklist 

Three  student  groups,  all  co-located,  were  observed  using  the  TLAC  Checklist. 
Distributed/collaborative  student  groups  were  not  observed  using  the  TLAC  Checklist  because 
students  during  this  phase  of  instruction  experienced  great  difficulty  with  the  wargaming  process 
due  to  inexperience  and  technical  difficulties  with  the  VTOC.  It  was  determined  that  it  would  be 
too  disruptive  to  the  students  to  ask  them  probe  questions  during  wargaming,  especially  since 


23 


they  seemed  highly  unlikely  to  be  able  to  answer  the  questions  effectively,  given  their  lack  of 
experience. 

Scores  on  the  TLAC  Checklist,  derived  by  averaging  the  quality  of  student  responses  to 
checklist  probe  questions  during  wargaming,  could  range  from  0-2.  Group  scores  on  the  TLAC 
Checklist  in  the  present  research  ranged  from  1.17  to  1.71,  suggesting  a  possible  restriction  of 
range.  However,  the  wargaming  of  all  student  groups  observed  was  rated  as  “above  average”  by 
their  instructor  (Instructor  A  for  all  three  groups),  which  suggests  that  the  range  restriction  on  the 
TLAC  Checklist  may  accurately  reflect  range  restriction  in  student  achievement.  Inter-rater 
reliability  for  the  TLAC  Checklist  was  not  assessed,  due  to  the  lack  of  secondary  raters  present 
during  the  observation  of  co-located  wargaming  sessions. 

As  with  the  Team  Communication  Checklists,  the  TLAC  Checklist  was  largely 
unobtrusive  during  co-located  observation.  Students  found  the  probe  questions  interesting  and 
were  more  than  willing  to  answer  them  during  wargaming.  However,  concerns  about  disrupting 
the  class  led  to  the  decision  not  to  ask  the  probe  questions  to  the  student  groups  as  a  whole  but  to 
individuals  in  the  group,  one  at  the  time.  Although  this  approach  bears  greater  similarity  to  the 
adaptive-thinking  training  approach  described  in  Lussier  et  al.  (2003),  it  does  not  truly  capture 
the  collective  adaptivity  of  team  thought,  even  though  individual  student  scores  are  averaged  to 
create  a  group  score. 

Situation  Awareness  (SA)  Exercise 

The  SA  Exercise  was  administered  to  three  student  groups,  all  distributed/collaborative, 
with  12  of  15  students  completing  the  exercise.  Three  students  did  not  complete  the  exercise  due 
to  technical  difficulties  with  the  VTOC.  The  SA  Exercise  was  not  administered  to  co-located 
groups  because  the  course  of  action  statement/sketch  to  be  used  during  wargaming  was 
necessary  to  design  the  SA  Exercise  and  one  was  not  available  for  the  1-22  CAV  mission 
scenario.  That  is,  student  groups  in  the  distributed/collaborative  phase  of  instruction  each  tended 
to  derive  the  same  course  of  action  from  the  TF  1-93  scenario,  but  students  in  the  co-located 
phase  of  instruction  did  not  (there  was  more  detailed  commander’s  guidance  in  the  TF  1-93 
scenario  than  the  1-22  CAV  scenario).  The  SA  Exercise  was  derived  from  a  course  of  action 
created  by  distributed/collaborative  students  who  were  observed  early  in  data  collection  and  who 
were  rated  as  good  wargamers  by  their  instructor.  For  this  reason,  the  first  two  groups  of 
distributed/collaborative  wargamers  observed  were  not  administered  the  SA  Exercise. 

The  average  individual  score  on  the  SA  Exercise  was  4.00,  with  a  range  of  0.00  to  7.00. 
The  average  team  score  was  1 .50,  with  a  range  of  .67  to  2.01 .  Given  the  possible  range  in  both 
individual  and  team  scores  of  0  to  30,  the  present  scores  hover  close  to  floor.  This  is  not 
surprising,  given  that  (a)  even  experienced  staffs  have  trouble  deriving  a  decision  support  matrix 
(on  which  the  SA  Exercise  is  based)  from  the  wargame;  and  (b)  the  students  in  the  present 
research  were  very  inexperienced.  As  an  illustration,  every  student  group  observed  was 
collectively  confused  about  the  basic  doctrinal  distinction  between  a  decisive  point  (a  point  on 
the  battlefield  where  decisive  results  must  be  achieved)  and  a  decision  point  (a  point  on  the 
battlefield  where  a  decision  must  be  made). 
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Simplifications  of  the  SA  Exercise  could  involve  reducing  the  questions  to  include  only 
Level  1  SA  (i.e.,  questions  that  require  students  to  identify  elements  of  the  present  situation). 

The  assumption  behind  this  approach  is  that  Level  1  SA  is  not  dependent  on  Level  2  and  Level  3 
SA,  which  is  an  untested  and  possibly  faulty  assumption  in  the  case  of  wargaming.  That  is,  in 
order  to  determine  the  elements  of  a  mission  plan,  wargamers  must  already  know  their 
implications  and  associated  future  activities.  A  second  possibility  would  be  to  conduct  the 
assessment  interactively  with  the  level  of  interactivity  decreasing  as  the  effectiveness  of  student 
responding  increases.  Recall  that  the  SA  Exercise  has  five  clusters  of  three  questions  with  each 
question  in  a  cluster  corresponding  to  one  of  the  three  levels  of  SA.  In  this  situation,  the 
instructor  would  begin  by  asking  the  student(s)  the  Level  1  SA  question  featured  in  the  first 
cluster,  probe  students  to  think  collaboratively  until  a  shared,  accurate  solution  is  reached, 
facilitate  student  identification  of  the  causes  of  inaccuracies,  then  move  on  to  the  next  level  of 
SA  in  that  cluster.  As  students  demonstrate  improved  collaborative  thinking  processes  and  SA, 
the  instructor  would  gradually  remove  himself  from  the  process,  letting  the  students  answer  the 
remaining  questions  on  their  own.  The  assessment  conducted  in  this  manner  initially  would 
show  low  levels  of  responding,  but  could  serve  as  a  useful  educational  tool  that  would  improve 
levels  of  responding  on  later  portions  of  the  assessment. 

Instructor  Ratings 

Two  instructors  assigned  quality  ratings  to  student  wargaming,  however,  no  student 
group  was  rated  by  both  instructors.  Collectively,  the  instructors  used  the  entire  scale  of  ratings 
(i.e..  Poor  to  Excellent,  Below  Average  to  Above  Average)  to  characterize  student  performance. 
Although  it  could  not  be  evaluated  formally,  Instructor  B  appears  to  have  been  slightly  harsher  in 
his  ratings  than  Instructor  A.  In  addition,  instructors  rated  student  wargaming  during  the  resident 
phase  of  instruction  higher  than  they  rated  student  wargaming  during  the 
distributed/collaborative  phase.  Student  groups  during  the  resident  phase  were  rated  either  as 
“good”  or  “excellent”  and  “average”  or  “above  average,”  whereas  groups  during  the 
distributed/collaborative  phase  were  rated  as  either  “good,”  “fair,”  or  “poor”  (4  of  5  “fair”  or 
“poor”)  and  “average”  or  “below  average.” 

Assessment  Validity.  This  section  summarizes  the  relation  of  each  assessment  to  one 
another,  together  with  the  implications  of  these  relations  for  assessment  validity  and  the  validity 
of  the  wargaming  framework  more  generally. 

Relationships  among  Individual  and  Team-Related  Wargaming  Determinants 

The  wargaming  framework  posits  several  possible  individual  and  team-related 
determinants  of  wargaming  process  and  outcome  effectiveness.  The  relative  importance  of  these 
determinants  in  accounting  for  individual  or  group  differences  in  wargaming  effectiveness  is  a 
function  of  the  degree  of  overlap  among  these  determinants.  That  is,  if  the  determinants  are 
highly  related  to  one  another,  any  one  determinant  is  less  useful  for  accounting  for  performance. 
Therefore,  the  relation  among  wargaming  determinants  presented  in  the  framework  is  expected 
to  be  low  to  moderate. 
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Of  the  determinants  presented  in  the  framework,  four  were  selected  for  the  development 
of  assessments,  and  three  of  these  assessments  (the  Staff  Roles  Knowledge  Assessment,  the 
Wargaming  Tacit  Knowledge,  Assessment,  and  the  Team-Motivation  Survey)  were  completed 
by  10  or  more  students.  In  addition,  several  demographic  variables  were  assessed.  Evaluating 
the  relation  between  the  wargaming  determinant  assessments  and  demographic  variables  sheds 
light  on  the  construct  validity  of  the  assessment  scores.  The  following  four  tables  summarize  the 
scores  on  the  assessments  of  wargaming  determinants  as  a  function  of  four  demographic 
variables:  rank,  company  command  experience,  staff  experience,  and  years  in  the  regular 
Army. 


As  shown  in  Table  8,  rank  does  not  have  an  apparent  relation  to  staff  roles 
knowledge  [t lieutenants, captainsO?)  =  .205,  j o  =  .839,  d  =  .07],  but  may  be  associated  with  wargaming 
tacit  knowledge  and  team-related  motivation.  Lieutenants  showed  higher  levels  of  wargaming 
tacit  knowledge  than  captains.  This  difference  was  statistically  non-significant  [f(24)  =1.186 ,p 
=  .247],  however,  the  effect  size  of  this  difference  approached  moderate  ( d  =  .46).  The  size  of 
this  effect  suggests  that  lieutenants,  on  average,  should  appear  to  an  educated  observer  (i.e., 
without  special-purpose  assessment)  to  have  a  more  nuanced  understanding  of  wargaming  than 
captains.  Moreover,  this  effect  size  is  especially  large  for  a  new  area  of  research  inquiry  in  a 
relatively  uncontrolled  experimental  setting  (see  Cohen,  1988,  pp.  25-27).  Team-related 
motivation  shows  an  increasing  trend  with  rank,  which  may  correspond  to  increased  investment 
in  the  Army,  but  this  trend  should  be  interpreted  with  great  caution  as  it  is  not  statistically 
significant  and  only  one  major  is  included  in  the  sample. 

Table  8 


Wargaming  Determinants  Scores  and  Officer  Rank 


LT 

CPT 

MAJ 

Staff  Roles  Knowledge 

.07  (W=14) 

.03  (JV=  20) 

.01  (W=l) 

Wargaming  Tacit  Knowledge 

2.05  (W=ll) 

1.33  (jV=15) 

N/A 

Team-Related  Motivation 

35.92  (N= 12) 

36.68  (iV=19) 

41  (N= 1) 

It  may  seem  surprising  that  increases  in  rank  are  not  associated  with  corresponding 
increases  in  staff  roles  knowledge,  but  it  should  be  remembered  that  the  students  in  the  course 
were  similarly  inexperienced  with  regard  to  staff  structure  and  function.  In  part,  the  purpose  of 
the  AC3DL  is  to  address  this  knowledge  gap.  For  this  reason,  it  is  unknown  why  lieutenants  on 
average  scored  higher  than  captains  on  the  Wargaming  Tacit-Knowledge  Assessment.  Prior  to 
enrolling  in  AC3DL,  neither  lieutenants  nor  captains  would  be  expected  to  have  a  great  deal  of 
experience  with  wargaming,  explicit  or  otherwise. 

As  shown  in  Table  9,  company  command  experience  does  not  appear  to  be  associated 
with  higher  scores  on  the  Team-Related  Motivation  Survey.  Company  command  experience, 
however,  appears  associated  with  slightly  higher  scores  on  the  Staff  Roles  Knowledge 
Assessment  and  the  Wargaming  Tacit  Knowledge  Assessment  [f(34)  =  -.868  and  t(23)  =  -.370, 
respectively].  The  differences  are  not  significant  but  the  relatively  small  effect  sizes  (d=  .31  and 
.15,  respectively),  may  be  expected  for  new  areas  of  research  inquiry  in  an  uncontrolled  research 
setting  (Cohen,  1988).  In  addition,  one  would  not  expect  great  differences  in  levels  of 
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knowledge  because  the  students  in  the  course  were  similarly  inexperienced  with  regard  to  staff 
structure  and  function  and  wargaming. 

Table  9 

Wargaming  Determinants  and  Company  Command  Experience 


Company  Command 
(CO  Cmd)  Experience  -  Yes 

CO  Cmd  Experience  -  No 

Staff  Roles  Knowledge 

.07  (TV =14) 

-.08  (TV  =22) 

Wargaming  Tacit 
Knowledge 

1.73  (TV  =10) 

1.52  (TV =15) 

Team-Related  Motivation 

36.58  (TV  =12) 

36.53  (TV  =19) 

The  data  in  Table  10  suggest  that  staff  experience  is  associated  with  increased  staff  roles 
knowledge  and  wargaming  tacit  knowledge,  as  would  be  expected.  The  difference  in  staff  roles 
knowledge  between  students  with  and  without  staff  experience  is  non-significant  [7(34)  =  -.993, 
p  =  .328],  but  the  effect  size  for  this  difference  (d  =  .34)  is  noteworthy  given  the  uncontrolled 
experimental  conditions.  The  difference  in  wargaming  tacit  knowledge  between  students  with 
and  without  staff  experience  is  non-significant  [/(23)  =  -.410,/?  =  .686],  but  the  effect  size  for 
this  difference  is  also  small  (d=  .17).  Staff  experience  appears  more  strongly  associated  with 
higher  levels  of  team-related  motivation  (7(29)  =  -1 .246,  p  =  .223,  d  =  .46].  This  effect  size  is 
substantial  in  light  of  the  relatively  uncontrolled  experimental  setting,  suggesting  that  officers 
with  staff  experience  should  appear  to  the  educated  observer  (i.e.,  without  using  a  special- 
purpose  assessment)  to  have  higher  levels  team-related  motivation. 

Table  10 

Wargaming  Determinants  and  Staff  Experience 


Staff  Experience  -  Yes 

Staff  Experience  -  No 

Staff  Roles  Knowledge 

.04  (TV  =22) 

-.12  (TV =14) 

Wargaming  Tacit 

Knowledge 

1.71  (TV =14) 

1.45  (TV =11) 

Team-Related  Motivation 

37.26  (TV =19) 

35.42  (TV =12) 

Table  1 1  shows  that  regular  Army  experience  is  not  associated  with  greater  levels  of  staff 
roles  knowledge  [/( 35)  =  -.205, p  =  .839 ,d=  .07].  However,  regular  Army  experience  does 
appear  associated  with  wargaming  tacit  knowledge  and  team-related  motivation.  The  difference 
in  wargaming  tacit  knowledge  between  students  with  and  without  regular  Army  experience, 
however,  is  non-significant  [/(24)  =  -.123 ,p  =  .903]  and  the  effect  size  for  this  difference  is  quite 
small  ( d  =  .05).  The  difference  in  team-related  motivation  between  students  with  and  without 
regular  Army  experience  is  non-significant  (7(30)  =  -1.691  ,p=  .100],  however  the  effect  size  is 
moderate-large  ( d  =  .60)  especially  given  the  uncontrolled  nature  of  the  experimental  setting.  It 
is  unclear  why  regular  Army  experience  would  be  so  strongly  related  to  team-related  motivation, 
but  perhaps  the  association  reflects  greater  investment  in  the  Army  in  general. 
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Table  1 1 

Wargaming  Determinants  and  Regular  Army  Experience 


Reg.  Army  Experience  -  Yes 

Reg.  Army  Experience - 
No 

Staff  Roles  Knowledge 

.01  (A  =22) 

-.02  (N =  15) 

Wargaming  Tacit  Knowledge 

1.67  (A  =  15) 

1.59  (A=  11) 

Team-Related  Motivation 

37.50  (A =  18) 

35.14  (A=  14) 

In  summary,  as  one  might  expect,  rank  and  regular  Army  experience  are  not  strongly 
associated  with  staff  roles  knowledge  and  wargaming  tacit  knowledge.  However,  where  a  trend 
(albeit  non-significant)  exists,  it  goes  in  the  direction  one  might  expect- greater  experience  in 
general  is  associated  with  higher  knowledge  scores-and  the  size  of  the  effects  found  generally 
correspond  to  those  that  would  be  expected  for  new  research  in  a  relatively  uncontrolled  setting. 
One  exception  is  the  association  between  rank  and  scores  on  the  Wargaming  Tacit  Knowledge 
Assessment,  where  for  unknown  reasons  higher  rank  is  associated  with  lower  scores.  Staff 
experience  and  company  command  experience  show  similar  trends,  with  experience  associated 
with  higher  scores  on  the  knowledge  assessments.  These  trends  show  stronger  effect  sizes  than 
those  generally  associated  with  rank  or  regular  Army  experience,  even  though  it  is  likely  that 
students  in  AC3DL  with  staff  experience  have  not  had  a  great  deal  of  formal  training  in  staff 
function  (Thompson,  Thompson,  Pleban  &  Valentine,  1991). 

Staff  experience  and  regular  Army  experience  do  appear  associated  with  team-related 
motivation,  however,  there  is  no  reason  to  expect  that  they  should  be.  It  is  possible  that  greater 
investment  in  the  Army  (as  reflected  in  having  greater  experience)  engenders  higher  levels  of 
motivation.  The  above  findings  suggest  that  the  assessments  of  wargaming  determinants 
developed  in  the  present  research  are  sensitive  to  experiential  variables  yet  also  reflect  the  lack 
of  variability  in  staff  and  wargaming  experience  present  among  AC3DL  students. 

A  final  test  of  the  assessments  of  wargaming  determinants  is  their  relation  to  one  another. 
It  is  expected  that  scores  on  these  assessments  will  show  a  weak  relation  to  one  another  as  they 
are  posited  to  contribute  uniquely  to  wargaming  process  and  outcome  effectiveness.  The  results 
of  this  test  are  shown  in  Table  12  below. 

Table  12 

Wargaming  Determinants  Relation  to  One  Another 


1. 

2. 

3. 

1.  Staff  Roles  Knowledge 

1.00  (A  =37) 

2.  Wargaming  Tacit  Knowledge 

.07  (A  =26) 

1.00  (A  =26) 

3.  Team-Related  Motivation 

-.01  (A  =32) 

-.24  (A  =23) 

1.00  (A  =32) 

Somewhat  surprisingly,  the  correlation  matrix  shown  in  Table  12  does  not  show  positive 
manifold  as  is  regularly  demonstrated  among  ability  test  correlations.  Instead,  two  of  the  three 
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correlation  coefficients  among  the  three  assessments  of  wargaming  determinants  are  negative, 
albeit  weak  and  non-significant.  It  is  unknown  why  these  correlations  are  negative,  but  the 
correlations  are  weak,  as  expected,  despite  the  satisfactory  reliabilities  of  these  assessments. 

Wargaming  Determinants  -  Relation  to  Wargaming  Process  Effectiveness 

The  wargaming  framework  posits  two  wargaming  processes:  team  communication  and 
adaptivity  of  team  thought.  It  is  expected  that  wargaming  determinants  will  show  a  positive 
relation  to  these  processes  but  that  these  processes  will  have  a  low  to  moderate  correlation  with 
one  another.  Both  of  the  wargaming  processes  posited  in  the  wargaming  framework  were 
assessed  in  the  present  research.  However,  they  were  not  assessed  at  the  same  time,  which 
means  that  they  were  also  not  assessed  during  wargaming  that  involved  the  same  mission 
scenario.  Moreover,  only  two  groups  of  students  were  observed  using  both  the  Team 
Communication  and  TLAC  Checklists.  For  these  reasons,  the  relation  between  team 
communication  and  adaptivity  of  team  thought  was  not  explored  in  the  present  research.  The 
relation  between  wargaming  determinants  and  processes  was  explored  and  the  findings  of  this 
exploration  are  shown  in  the  following  three  tables. 

Table  13  below  shows  the  average  Staff  Roles  Knowledge  Assessment  score  for  each 
group  of  students,  the  same  score  for  each  group’s  executive  officer  (XO),  or  team  lead,  and  the 
corresponding  wargaming  process  scores.  Where  a  particular  group  was  not  observed  using  one 
of  the  checklists,  “N/A”  is  indicated  in  the  corresponding  cell.  Where  a  group  was  observed 
using  both  checklists  (i.e.,  the  group  was  observed  in  both  the  distributed/collaborative  and 
resident  phases  of  AC3DL,  groups  2  and  4),  there  are  two  Staff  Roles  Knowledge  Assessment 
scores  representing  differences  in  the  group  composition  between  the  distributed/collaborative 
and  resident  phases  of  instruction.  For  ease  of  review,  groups  are  listed  in  ascending  order 
according  to  their  Staff  Roles  Knowledge  Assessment  Score. 

Table  13 

Staff  Roles  Knowledge  and  Wargaming  Process  Effectiveness 


Group 

Individual  Determinant 

Process  Effectiveness 

Staff  Roles  Knowledge 

Team  Communication 

TLAC  Checklist 

Group  Avg. 

XO  Score 

Total 

Task 

Info  Share 

1 

-.24 

-.04 

.53 

.67 

.38 

N/A 

4a 

-.17 

-.63 

.28 

.46 

.15 

N/A 

6 

-.04 

.32 

.29 

.41 

.17 

N/A 

4b 

.03a 

-.63 

N/A 

N/A 

N/A 

1.71 

3 

.01 

.01 

N/A 

N/A 

N/A 

1.56 

2a 

.10 

.40 

.42 

.55 

.29 

N/A 

5 

.20 

-.04 

.28 

.46 

.09 

N/A 

2b 

.24b 

.40 

N/A 

N/A 

N/A 

1.17 

“Four  of  7  group  members  completed  the  Staff  Roles  Knowledge  Assessment.  This  average  should  be  interpreted 
with  caution. 

bOnly  3  of  12  group  members  completed  the  Staff  Roles  Knowledge  Assessment.  Eight  students  did  not  complete 
the  assessment  as  they  were  not  present  in  a  distributed/collaborative  session  that  was  observed  during  data 
collection.  One  student  did  not  complete  the  assessment  for  unknown  reasons.  This  average  should  not  be 
considered  representative  of  the  group  as  a  whole. 
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Average  scores  on  the  Staff  Roles  Knowledge  Assessment  do  not  appear  clearly  linked  to 
wargaming  process  effectiveness  scores.  Where  wargaming  process  effectiveness  is  assessed  via 
the  TLAC  Checklist,  there  are  too  few  groups  assessed  to  begin  looking  for  a  pattern.  Where 
wargaming  process  effectiveness  is  assessed  via  the  Team  Communication  Checklist,  a  clear 
pattern  linking  Team  Communication  total  scores  and  staff  roles  knowledge  is  not  apparent.  The 
same  is  true  when  the  Staff  Roles  Knowledge  Assessment  score  for  each  group’s  executive 
officer  (i.e.,  team  lead)  is  compared  to  Team  Communication  total  scores.  A  pattern  is  more 
readily  seen  when  each  team  leader’s  Staff  Roles  Knowledge  Assessment  score  is  compared  to 
the  Team  Communication  information  sharing  score.  Here,  higher  levels  of  staff  roles 
knowledge  generally  are  associated  with  higher  information  sharing  scores.  When  examining 
these  patterns,  however,  it  should  be  recalled  that  Group  1  was  observed  for  Team 
Communication  during  co-located  wargaming  with  a  less  central  recording  technique  and  better 
simulation  of  the  operational  environment  (see  pp.  23-24).  In  addition,  the  restriction  in  range 
in  Staff  Roles  Knowledge  Assessment  scores  may  reduce  the  meaningfulness  of  the  associations 
reviewed  here.  The  sample  size  (i.e.,  number  of  groups)  is  too  small  to  compute  a  reliable 
correlation. 

Table  14  below  shows  the  average  Wargaming  Tacit  Knowledge  (TK)  Assessment  score 
for  each  group  of  students,  the  same  score  for  each  group’s  XO,  and  the  corresponding 
wargaming  process  scores.  Where  a  particular  group  was  not  observed  using  one  of  the 
checklists,  “N/A”  is  indicated  in  the  corresponding  cell.  Where  a  group  was  observed  using  both 
checklists  (i.e.,  the  group  was  observed  in  both  the  distributed/collaborative  and  resident  phases 
of  AC3DL,  groups  2  and  4),  there  are  two  Wargaming  Tacit  Knowledge  Assessment  scores 
representing  differences  in  the  group  composition  between  the  distributed/collaborative  and 
resident  phases  of  instruction.  For  ease  of  review,  groups  are  listed  in  ascending  order  according 
to  their  Wargaming  Tacit  Knowledge  Assessment  Score. 

Table  14 

Wargaming  Tacit  Knowledge  and  Wargaming  Process  Effectiveness 


Group 

Individual  Determinant 

Process  Effectiveness 

Wargaming  TK 

Team  Communication 

TLAC  Checklist 

XO  Score 

Total 

Task 

Info  Share 

4a 

.33 

1.00 

.28 

.46 

.15 

N/A 

4b 

1.19“ 

1.00 

N/A 

N/A 

N/A 

1.71 

5 

1.88 

2.50 

.28 

.46 

.09 

N/A 

6 

1.25 

.00 

.29 

.41 

.17 

1.71 

1 

2.00 

.00 

.53 

.67 

.38 

N/A 

2a 

2.13 

2.50 

.42 

.55 

.29 

N/A 

2b 

1.33b 

2.50 

N/A 

N/A 

N/A 

1.17 

3 

N/A 

N/A 

N/A 

N/A 

N/A 

1.56 

“Four  of  7  group  members  completed  the  Wargaming  Tacit  Knowledge  Assessment.  This  average  should  be 
interpreted  with  caution. 

bOnly  3  of  12  group  members  completed  the  Wargaming  Tacit  Knowledge  Assessment.  Eight  students  did  not 
complete  the  assessment  as  they  were  not  present  in  a  distributed/collaborative  session  that  was  observed  during 
data  collection.  One  student  did  not  complete  the  assessment  for  unknown  reasons.  This  average  should  not  be 
considered  representative  of  the  group  as  a  whole. 
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Table  14  shows  a  generally  increasing  trend  in  both  average  group  wargaming  tacit 
knowledge  scores  and  Team  Communication  total  scores.  The  average  wargaming  tacit 
knowledge  score  for  student  groups  scoring  lowest  in  team  communication  is  1.15,  whereas  it  is 
2.07  for  groups  scoring  highest  in  team  communication.  There  appears  to  be  no  association 
between  the  tacit-knowledge  score  of  the  student  group  team  lead  (XO)  and  wargaming  process 
effectiveness  as  captured  by  either  checklist.  The  sample  size  (i.e.,  number  of  groups)  is  too 
small  to  compute  a  reliable  correlation. 

Table  15  below  shows  the  average  Team-Related  Motivation  Survey  score  for  each  group 
of  students  and  their  corresponding  wargaming  process  scores.  As  with  the  previous  two  tables, 
where  a  particular  group  was  not  observed  using  one  of  the  checklists,  “N/A”  is  indicated  in  the 
corresponding  cell.  Where  a  group  was  observed  using  both  checklists  (i.e.,  the  group  was 
observed  in  both  the  distributed/collaborative  and  resident  phases  of  AC3DL),  there  are  two 
Team-Related  Motivation  Survey  scores  representing  differences  in  the  group  composition 
between  the  distributed/collaborative  and  resident  phases  of  instruction.  Also  similar  to  the 
previous  two  tables,  groups  are  listed  in  ascending  order  according  to  their  Team-Related 
Motivation  Survey  score. 

Table  15 

Team-Related  Motivation  and  Wargaming  Process  Effectiveness 


Group 

Individual  Determinant 

Process  Effectiveness 

Team-Related  Motivation 

Team  Communication 

TLAC  Checklist 

Group  Avg. 

Total 

Task 

Info  Share 

5 

33.20 

.28 

.46 

.09 

N/A 

3 

35.33 

N/A 

N/A 

N/A 

1.56 

6 

36.33 

.29 

.41 

.17 

N/A 

2a 

37.25 

.42 

.55 

.29 

N/A 

2b 

N/A 

N/A 

N/A 

1.17 

1 

38.20 

.53 

.67 

.38 

N/A 

4a 

41.67 

.28 

.46 

.15 

N/A 

4b 

39.25b 

N/A 

N/A 

N/A 

1.71 

“Only  1  of  12  group  members  completed  the  Team-Related  Motivation  Survey.  Eight  students  did  not  complete  the 
assessment  as  they  were  not  present  in  a  distributed/collaborative  session  that  was  observed  during  data  collection. 
Three  students  did  not  complete  the  assessment  for  unknown  reasons.  This  average  should  not  be  considered 
representative  of  the  group  as  a  whole. 

bFour  of  7  group  members  completed  the  Team-Related  Motivation  Survey.  This  average  should  be  interpreted  with 
caution. 


It  is  difficult  to  discern  a  pattern  in  the  results  this  table.  The  student  groups  with  the 
highest  and  lowest  motivation  scores  performed  similarly  with  regard  to  team  communication. 
One  possible  explanation  for  the  lack  of  a  clear  pattern  (aside  from  the  small  sample  size)  is 
range  restriction  in  the  motivation  survey  scores.  Because  the  possible  range  in  scores  on  the 
survey  is  0-48,  the  observed  range  of  33.20-41.67  may  not  represent  meaningful  variation  (i.e.,  a 
score  of  33.20  represents  an  average  endorsement  of  2.67  using  a  scale  of  0-4  and  a  score  of 
41.67  represents  an  average  endorsement  of  3.47). 
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In  summary,  the  results  are  mixed  regarding  the  relation  between  the  wargaming 
determinants  identified  in  the  conceptual  framework  and  wargaming  process  effectiveness.  The 
staff  roles  knowledge  score  earned  by  the  student  group  team  lead  (XO)  and  the  average  group 
wargaming  tacit-knowledge  score  appear  associated  with  the  quality  of  team  communication, 
although  further  exploration  with  a  larger  number  of  student  groups  is  required  to  draw  firm 
conclusions.  Team-related  motivation  was  consistently  high  across  groups,  and  thus  did  not 
appear  associated  with  the  quality  of  team  communication.  Unfortunately,  there  were  too  few 
administrations  of  the  TLAC  Checklist  (given  its  requirement  for  student  expertise  and  dedicated 
class  time)  to  reliably  evaluate  the  relation  between  wargaming  determinants  and  adaptivity  of 
team  thought. 

When  reviewing  the  above  tables,  one  should  bear  in  mind  several  possible  explanations 
for  why  a  clearer  pattern  of  results  was  not  observed.  The  first  reason  is  that  the  number  of 
groups  observed  was  relatively  small.  Although  all  of  the  AC3DL  student  groups  that  could 
have  been  observed  were  observed,  six  is  too  small  a  number  to  reliably  identify  a  pattern  in  the 
data.  Moreover,  no  more  than  five  groups  was  observed  with  any  one  checklist.  Second,  5  of 
the  6  groups  was  comprised  of  six  or  fewer  students,  making  average  scores  unstable.  Third, 
there  were  differences  in  the  conditions  under  which  team  communication  was  observed.  These 
differences  include  the  wargaming  environment  (co-located  or  distributed),  the  number  of 
students  wargaming,  the  extent  of  technical  problems  experienced  during  the  war  game,  etc. 
These  situational  differences  compete  with  individual/group  differences  in  determining  the 
effectiveness  of  wargaming  processes. 

Wargaming  Determinants  and  Process  Effectiveness  -  Relation  to  Wargaming  Outcome 
Effectiveness 

The  wargaming  framework  posits  two  wargaming  outcomes:  shared  battlefield 
visualization  and  an  integrated  mission  plan.  It  is  expected  that  wargaming  processes  will  show 
a  positive  relation  to  these  outcomes  but  that  these  outcomes  will  be  moderately  associated  with 
one  another.  Only  one  of  the  wargaming  outcomes  posited  in  the  wargaming  framework  was 
assessed  in  the  present  research.  Therefore,  the  relation  between  shared  battlefield  visualization 
and  the  level  of  integration  in  the  mission  plan  could  not  be  explored.  The  relation  between 
wargaming  determinants,  processes,  and  outcomes  was  explored  and  the  findings  of  this 
exploration  are  shown  in  the  following  three  tables.  In  addition,  instructor  ratings  were  used  as 
outcome  assessments  to  explore  external  validity. 

Table  16  below  shows  the  wargaming  process  scores  for  each  group  of  students  and  their 
corresponding  wargaming  outcome  score  and  instructor  ratings.  Where  a  particular  group  was 
not  observed  or  did  not  complete  the  SA  Exercise,  “N/A”  is  indicated  in  the  corresponding  cell. 
The  ratings  in  the  instructor  rating  columns  should  be  interpreted  as  “distributed  wargaming/co¬ 
located  wargaming.”  Where  wargaming  was  observed  in  only  one  environment,  that 
environment  is  listed  in  parentheses  next  to  the  instructor  rating,  with  “(d)”  indicating  distributed 
and  “(c)”  indicating  co-located.  For  ease  of  review,  groups  are  listed  in  ascending  order 
according  to  their  Team  Communication  score. 
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Table  16 


Wargaming  Processes  and  Wargaming  Outcomes 


Wargaming  Process  Effectiveness 

Wargaming 

Outcome  Effectiveness 

Group 

Team  Communication 

TLAC 

Checklist 

SA  Exercise 

Instructor  Rating 
-  Doctrine 

Instructor 

Total 

Task 

Info 

Share 

Average 

Individual 

Team 

Score 

Rating  - 
Others 

5 

.28 

.46 

.09 

N/A 

3.26 

3.33 

Poor  (d) 

Below  Avg. 
(d) 

6 

.29 

.41 

.17 

N/A 

4.00 

1.50 

Fair  (d) 

Avg.  (d) 

4 

.28 

.46 

.15 

1.71 

2.00 

.67  . 

Fair/Good 

Avg./Above 

Avg. 

2 

.42 

.55 

.29 

1.17 

N/A 

N/A 

Good/Excellent 

Avg./Above 

Avg. 

1 

.53 

.67 

.38 

N/A 

N/A 

N/A 

'iiT-r-mnr 

Average  (c) 

3 

N/A 

N/A 

N/A 

1.56 

N/A 

N/A 

Good  (c) 

Above  Avg. 
(c) 

“This  group  was  also  observed  during  distributed  wargaming,  but  no  wargaming  process  observations  were 
conducted  at  that  time,  so  the  ratings  are  left  out  to  aid  in  interpretation. 


Table  16  generally  shows  that  as  Team  Communication  scores  increase,  instructor  ratings 
also  increase.  On  average,  student  groups  rated  “fair”  or  “poor”  earned  Team  Communication 
scores  of  .28  (Total),  .44  (Task),  and  .14  (Info  Share),  whereas  student  groups  rated  “good”  or 
“good/excellent”  earned  average  Team  Communication  scores  of  .48  (Total),  .61  (Task),  and  .34 
(Info  Share).  Unfortunately,  there  are  too  few  data  points  for  the  SA  Exercise  to  reliably 
evaluate  the  relation  between  this  wargaming  outcome  and  wargaming  processes. 

Table  17  below  shows  the  average  individual  and  team-related  wargaming  determinants 
scores  for  each  group  of  students  and  their  corresponding  wargaming  outcome  score  and 
instructor  ratings.  Where  a  particular  group  did  not  complete  an  assessment,  “N/A”  is  indicated 
in  the  corresponding  cell.  Where  wargaming  was  observed  in  only  one  environment,  that 
environment  is  listed  in  parentheses  next  to  the  instructor  rating,  with  “(d)”  indicating  distributed 
and  “(c)”  indicating  co-located.  For  groups  who  wargamed  in  both  distributed  and  co-located 
environments  (Groups  1, 2,  and  4),  there  are  two  sets  of  wargaming  determinant  scores 
representing  differences  in  the  group  composition  between  the  distributed/collaborative  and 
resident  phases  of  instruction2.  For  ease  of  review,  student  groups  are  listed  in  ascending  order 
according  to  their  instructor  ratings. 


Table  17  generally  shows  that  as  instructor  ratings  increase,  scores  on  the  wargaming 
determinants  assessments  roughly  increase  as  well.  For  student  groups  rated  “fair”  or  “poor”  by 
instructors,  the  average  Staff  Roles  Knowledge  Assessment  score  was  .00,  whereas  for  student 
groups  rated  “good”  of  “excellent”  by  instructors,  this  was,  on  average,  .03.  The  average  Staff 
Roles  Knowledge  Assessment  score  of  the  team  leads  (XOs)  for  groups  rated  “fair”  or  “poor” 
was  -.34  in  contrast  to  .03  for  groups  rated  “good”  or  “excellent.”  Similarly,  the  average 
wargaming  tacit  knowledge  score  for  lower-rated  (“poor”  or  “fair”)  student  groups  was  1.15, 
whereas  the  score  earned  by  the  higher-rated  (“good”  or  “excellent”)  groups  was  1 .66.  Once 


2  Recall,  however,  that  Group  1  did  not  receive  wargaming  determinant  assessments  before  distributed  wargaming. 
Therefore,  no  scores  are  available  and  N/A  is  shown  in  the  table. 
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again,  team-related  motivation  does  not  bear  a  clear  relationship  to  instructor  ratings  of 
wargaming  effectiveness  (37.07  for  lower-rated  groups  vs.  38.01  for  higher-rated  groups).  All 
of  these  differences  obviously  should  be  interpreted  with  caution,  however,  given  the  limited 
number  of  groups.  Unfortunately,  there  are  too  few  data  points  for  the  SA  Exercise  to  reliably 
evaluate  the  relation  between  this  wargaming  outcome  and  wargaming  determinants. 

In  summary,  it  appears  the  assessments  of  wargaming  determinants  and  processes  may  be 
related  to  wargaming  effectiveness  as  rated  by  instructors.  This  result  supports  hopeful 
conclusions  about  the  validity  of  the  wargaming  assessments,  of  the  wargaming  conceptual 
framework,  and  of  the  very  endeavor  to  assess  wargaming  performance.  Although  the  findings 
presented  in  this  section  must  be  greeted  with  the  caution  appropriate  when  very  small  sample 
sizes  and  an  uncontrolled  assessment  situation  are  involved,  the  several  lines  of  converging 
evidence  just  described  suggest  that  similar  results  might  be  found  with  larger  sample  sizes. 

This,  of  course,  remains  to  be  tested. 

Table  17 

Wargaming  Determinants  and  Wargaming  Outcomes 


Group 

Wargaming  Determinant  Scores 

Wargaming  Outcome  Effectiveness 

Staff  Roles 
Knowledge 

Wargaming 

TK 

Team- 

Related 

Motivation 

SA  Exercise 

Instructor 
Rating  - 
Doctrine 

Instructor 
Rating  - 
Others 

Group 

Avg. 

XO 

Group 

Avg. 

XO 

Avg 

Individual 

Team 

Score 

5 

.47 

.21 

1.88 

2.50 

33.20 

3.26 

3.33 

Poor  (d) 

Below  Avg. 
(d) 

la 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

Poor 

lb 

.39 

.36 

2.00 

.00 

38.20 

N/A 

N/A 

Good 

HBEXEESHi 

6 

.41 

.48 

1.25 

.00 

36.33 

4.00 

1.50 

■B' 

Average  (d)  | 

4a 

.42 

.30 

.33 

1.00 

41.67 

2.00 

.67 

Fair 

4b 

.43 

.30 

1.19 

1.00 

39.25 

N/A 

N/A 

Good8 

Above  Avg. 

3 

.42 

.39 

N/A 

N/A 

35.33 

N/A 

N/A 

Good  (c) 

Above 
Average  (c) 

2a 

.45 

.52 

2.13 

2.50 

37.25 

N/A 

N/A 

Good 

2b 

.46 

.52 

1.33 

2.50 

40.00 

N/A 

N/A 

Above  Avg. 

“Wargaming  determinant  scores  do  not  correspond  exactly  to  observations  during  co-located 
participants  in  the  co-located  wargaming  did  not  complete  the  wargaming  determinants  asses 
°Eight  participants  in  the  co-located  Wflroatnino  HiH  not  mmnlptp  thp  wornamino  Hpfprtninanl 


_ 0 _ wargaming.  Three 

wargaming  determinants  assessments. 


Discussion 

The  present  research  provides  a  more  in-depth  understanding  of  wargaming  and  its 
effectiveness  assessment  than  previously  has  been  accomplished.  Several  psychological 
constructs  comprising  wargaming  determinants,  processes,  and  outcomes  have  been  identified. 
In  addition,  the  feasibility  and  (where  applicable)  the  psychometric  soundness  of  assessments  of 
several  of  these  constructs  when  administered  to  groups  of  wargamers  in  an  uncontrolled 
environment  have  been  demonstrated.  This  work  therefore  sets  the  stage  for  more  systematic 
assessment  of  wargaming  effectiveness  and  diagnosis  of  shortfalls  in  wargaming  performance. 
Understanding  the  true  implications  of  the  present  research  and  designing  its  thoughtful 
extension,  however,  requires  consideration  of  its  strengths  and  limitations,  summarized  below. 
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Strengths 


The  fundamental  strength  of  the  present  research  is  its  contribution  to  understanding  and 
assessing  wargaming  effectiveness.  The  framework  for  conceptualizing  wargaming  is  based  on 
the  application  of  psychological  theory  to  understanding  operational  behavior,  thus  making  the 
framework  relatively  general  across  wargaming  situations,  acceptable  to  SMEs,  and 
understandable  to  psychologists  tasked  with  assessment  development.  In  addition,  the 
assessments  developed  are  flexible.  They  are  portable  across  wargaming  environments, 
including  co-located  and  distributed,  traditional  and  digitally  supported  wargaming 
environments.  They  also  can  be  implemented  in  either  paper-and-pencil  format  or  in 
computerized  format  with  automatic  data  recording  and  tabulation. 

Importantly,  the  assessments  also  are  portable  across  wargaming  situations  involving 
different  mission  scenarios.  The  assessments  of  wargaming  determinants  (individual  and  team- 
related)— the  MABE,  the  Staff  Roles  Knowledge  Assessment,  the  Wargaming  Tacit  Knowledge 
Assessment,  and  the  Team-Related  Motivation  Survey-may  be  reused  without  modification 
regardless  of  mission  scenario.  The  wargaming  process  and  outcomes  assessments— the  Team 
Communication  Checklists,  the  TLAC  Checklists,  the  SA  Exercise,  and  the  Integrated  Overlay 
Exercise— are  partially  portable  across  wargaming  situations.  Because  these  assessments  are 
scenario-specific,  their  portability  is  constrained  by  the  degree  of  similarity  among  the  mission 
scenarios  used  in  each  wargaming  situation.  However,  because  a  well-documented  and 
principled  approach  was  used  to  design  the  assessments,  the  modification  of  scenario-specific 
assessments  is  a  straightforward,  if  effortful  endeavor.  Appendix  J  provides  detailed  design 
guidelines  for  modifying  the  scenario-specific  assessments  developed  in  the  present  research. 

Limitations 

Although  the  present  research  provides  a  framework  for  conceptualizing  wargaming 
determinants  (individual  and  team-related),  process,  and  outcomes,  this  framework  has  not  been 
formally  validated  through  large-scale  correlational  study  and  external  validation.  Such  a  formal 
validation  effort  was  beyond  the  scope  and  resources  of  the  present  research  project,  but 
nevertheless  such  a  validation  should  be  conducted  before  the  framework  is  considered 
“official.”  The  primary  threat  to  the  validity  of  the  framework  is  the  lack  of  representativeness  of 
the  wargaming  environment  studied.  That  is,  the  research  was  conducted  using  officers  who 
were  simulating  staff  operations  as  part  of  a  learning  exercise.  These  officers  were  not  staff 
officers  and  had  very  little  experience  with  combined  arms  military  planning.  More  importantly, 
the  wargaming  observed  as  part  of  the  present  research  had  been  stripped  of  much  of  its 
operational  context  in  order  to  generate  a  feasible  learning  exercise.  Missing  context,  such  as 
(among  others)  vertical  coordination  with  brigade  staff  elements,  use  of  operational  command 
and  control  technology,  and  the  presence  of  a  commander,  may  have  resulted  in  an  overly 
simplified  framework.  The  framework  was  designed  with  the  intent  that  it  be  robust  across  the 
range  of  wargaming  situations  (e.g.,  experienced  vs.  inexperienced  staffs,  conventional  vs. 
unconventional  mission  planning,  etc.),  but  the  actual  stability  of  the  framework  remains  to  be 
tested. 
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A  second  limitation  of  the  present  research  concerns  the  validation  of  the  assessments 
that  were  developed  and  administered.  Here  too,  a  formal  validation  effort  was  beyond  the  scope 
and  resources  of  the  present  research  project.  The  assessments  could  be  administered  neither  to  a 
large  number  of  student  groups,  nor  to  research  participants  with  a  range  of  experience  with 
wargaming  and  military  planning.  Although  the  initial  results  are  promising,  a  more  controlled 
validation  study  is  required  to  formally  validate  the  assessments. 

Conclusions  and  Future  Directions 

In  the  present  research,  the  wargaming  process  was  explored  to  gain  a  more  in-depth 
understanding  than  has  previously  been  accomplished,  an  understanding  necessary  to  design 
reliable  and  valid  wargaming  performance  assessments.  This  exploration  supported  the  main 
research  objective  of  identifying,  developing,  and  validating  techniques  for  assessing  the 
determinants  (individual  and  team-related),  processes,  and  outcomes  of  effective  wargaming. 

An  important  lesson  learned  from  the  present  research  is  that  wargaming  assessment 
validation  efforts  must  involve  an  integrated  approach  in  which  assessment  design  is  based  on  a 
thorough  understanding  of  the  activity  to  be  assessed,  assessment  implementation  is  based  on  a 
thorough  understanding  of  the  environment  in  which  assessment  will  occur,  and  assessment 
validation  is  accomplished  in  a  controlled  setting  in  which  specific  hypotheses  about  the 
assessments  can  be  tested.  The  integrated  approach  enables  researchers  to  empirically  validate 
performance  assessments  while  at  the  same  time  evaluating  the  feasibility  of  the  assessments  and 
ensuring  their  relevance  in  the  actual  performance  environment. 

The  conditions  under  which  the  present  research  was  conducted  did  not  permit  work  with 
a  large  number  of  research  participants  in  a  controlled  setting,  as  is  recommended  for 
assessment-validation  studies.  Moreover,  because  the  research  involved  a  small  number  of 
student  groups  from  just  one  course  (AC3DL),  the  ability  to  field-test  certain  hypotheses  about 
the  validity  of  the  assessments  was  significantly  limited.  However,  it  was  possible  to  determine 
what  the  nature  of  assessments  for  capturing  wargaming  performance  should  be  (i.e.,  what 
competencies  and  behaviors  should  be  assessed)  and  to  evaluate  the  feasibility  of  implementing 
these  assessments  in  multiple  performance  environments. 

Important  technical  factors  to  consider  when  implementing  performance  assessments  in 
collaborative  virtual  environments  include  the  speed  of  the  Internet  connection  used  by  the 
examinees,  the  operating  system  and  service  packs  in  use  on  their  personal  computers,  and  the 
stability  of  the  virtual  environment.  The  reality  of  advanced  collaboration  platforms  is  that  they 
are  not  similarly  experienced  by  all  collaborators  and  that  difficulty  with  various  elements  of  the 
technology— due  to  unforeseen  and  non-replicable  conditions— is  commonplace.  These  factors 
influence  the  feasibility  of  all  types  of  assessments,  including  observer  checklists  and  automated 
data  collection.  To  ensure  that  assessments  capture  what  they  are  intended  to  capture,  designers 
and  developers  must  make  certain  that  the  assessments  will  be  accessible  to  the  range  of 
computer  configurations  available  to  examinees  and  robust  in  the  face  of  technical  difficulty. 

Important  behavioral  factors  to  consider  when  implementing  performance  assessments  in 
collaborative  virtual  environments  include  the  time  required  to  administer/score  the  assessments, 
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the  usability  of  the  assessments,  the  level  of  experience  of  the  individuals  taking  the  assessments, 
and  the  level  of  buy-in  with  which  both  examinees  and  other  interested  parties  approach  the 
assessment  methods  and  results.  These  factors  influence  whether  the  assessments  are  actually 
administered  and/or  whether  the  results  of  the  assessments  will  be  valued.  To  ensure  that 
assessments  will  be  used,  designers  and  developers  must  work  closely  with  stakeholders  to  make 
certain  the  assessments  (a)  capture  constructs  or  behaviors  of  interest;  (b)  feature  an  appropriate 
level  of  difficulty,  given  examinee  experience  levels;  (c)  require  minimal  learning  to 
administer/take;  and  (d)  produce  scores  and  feedback  that  are  meaningful  to  those  involved. 

Future  research  exploring  the  assessment  of  collective  staff  activity  during  military 
planning  should  therefore  focus  on  methods  of  integrating  task  and  performance-environment 
analysis  with  controlled  validation  study.  Controlled  study  would  allow  the  identification  of  best 
practice  in  assessment  design  and  implementation  applied  broadly  across  environments  in  which 
planning  is  conducted  (e.g.,  different  types  of  units,  differing  amounts  of  staff  turnover  or  staff 
expertise,  differing  levels  of  digitization).  Task  analysis  should  guide  the  design  and 
development  of  the  assessments  to  be  validated,  and  performance-environment  analysis  should 
illuminate  the  boundary  conditions  that  constrain  the  generalizability  of  the  validation  results. 

Where  wargaming  is  to  remain  the  subject  of  future  research,  study  exploring  the 
implications  of  transformation  for  the  integration  and  synchronization  of  the  battlefield  operating 
systems  is  most  needed.  That  is,  the  transformation  of  brigades  and  battalions  to  modular  units 
has  implications  for  the  composition  of  the  brigade  and  battalion  battle  staffs,  the  roles  of  their 
members,  and  the  nature  of  mission  plans.  One  example  of  significant  change  in  staff 
composition  and  roles  is  the  advent  of  the  fires  and  effects  coordination  cell  (FECC)  now  present 
in  the  brigade  staff  and  closely  linked  to  the  field  artillery  battalion  staff  ( FMI  3-09.42;  DA, 
2005).  The  FECC  differs  from  the  traditional  brigade  fire  support  element  in  that  it  features  an 
information  operations  component  which  aids  in  fires  and  effects  targeting  (Glenister,  2002). 

Yet,  as  the  absence  of  specific  doctrine  reflects,  the  tactics  for  arranging  assets  in  time  and  space 
to  conduct  effects-based  operations  against  the  asymmetric  enemy  are  largely  unknown,  and  the 
optimal  staff  composition  for  accomplishing  effects-based  mission  planning  is  yet  unachieved. 
Moreover,  the  nature  of  effects-based  operations,  which  the  FECC  is  designed  to  support,  is  not 
well  understood  by  staff  officers  or  their  commanders. 

The  approach  developed  in  the  present  research,  the  conceptual  framework  for 
wargaming,  and  the  guidelines  for  assessment  development  serve  as  a  springboard  for  future 
assessment  research  in  both  field  and  controlled  settings,  and  in  some  combination  of  the  two.  If 
extended  and  applied,  this  work  could  escort  operations  command  and  control  teams  into  the 
future  through  a  better  understanding  of  how  to  develop  and  support  their  collective  mission 
planning  competence. 
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ABF 

AC3 

AC3DL 

ACES 

ADA 

ADO 

ALO 

AO 

ARC 

ASR 

AT 

ATK 

AXP 

BDE 

BHL 

BN 

BOS 

BPT 

C2 

C4 

CABTF 

CALL 

CAS 

CASEVAC 

CAT 

CATK 

CCIR 

CDR 

CFL 

CGSC 

CHEMO 

CL 

CL  I 

CL  II 

CL  III 
CL  III  (B) 
CL  IV 
CL  V 
CL  VII 


Appendix  A 
List  of  Acronyms 
Assault  by  Fire 

Armor  Captain’s  Career  Course 

Armor  Captain’s  Career  Course  -  Distance  Learning 

Armored  Combat  Earthmovers 

Air  Defense  Artillery 

Air  Defense  Officer 

Air  Liaison  Officer 

Area  of  Operations 

Action-Reaction-Counteraction 

Alternate  Supply  Route 

Anti-Tank 

Attack 

Ambulance  Exchange  Point 
Brigade 

Battle  Handoff  Line 
Battalion 

Battlefield  Operating  Systems 
Be  Prepared  To 

Command  and  Control 

Command  Control  Communications  and  Computers 

Combined  Arms  Battalion  Task  Force 

Center  for  Army  Lessons  Learned 

Close  Air  Support 

Casualty  Evacuation 

Civil  Affairs  Team 

Counterattack 

Commander’s  Critical  Information  Requirements 

Commander 

Coordinated  Fire  Line 

Command  and  General  Staff  College 

Chemical  Officer 

Class 

Class  I:  Subsistence  Items 

Class  II:  Individual  Equipment,  Clothing,  Tool  Sets,  House 
Keeping  Supplies 

Class  III:  Petroleum,  Oils,  Lubricants 

Class  III  (b):  Bulk  Petroleum 

Class  IV :  Construction  and  Barrier  Material 

Class  V:  Ammunition 

Class  VII:  Major  End  Items 


A-l 


CL  VIII 

Class  VIII:  Medical  Material 

CL  IX 

Class  IX:  Repair  Parts  and  Components 

CO  CMD 

Company  Command 

COA 

Course  of  Action 

COLT 

Combat  Observation  Lasing  Team 

CP 

Command  Post 

CPT 

Captain 

CS 

Combat  Support 

CSOP 

Combat  Security  Outpost 

CSS 

Combat  Service  Support 

CTCP 

Combat  Trains  Command  Post 

DA 

Department  of  the  Army 

DP 

Decision  Point 

DSM 

Decision  Support  Matrix 

DST 

Decision  Support  Template 

EEFI 

Essential  Elements  of  Friendly  Information 

ELINT 

Electronics  Intelligence 

ENG 

Engineer 

EW 

Electronic  Warfare 

FA 

Field  Artillery 

FARP 

Forward  Arming  and  Refueling  Point 

FASCAM 

Family  of  Scatterable  Mines 

FECC 

Fires  and  Effects  Coordination  Cell 

FFIR 

Friendly  Forces  Information  Requirements 

FFTLOK 

Fighters  for  the  Liberation  of  Kentuckia 

FIST 

Fire  Integration  Support  Team 

FLOT 

Forward  Line  of  Own  Troops 

FM 

Field  Manual 

FMI 

Field  Manual  -  Interim 

FSC 

Forward  Support  Company 

FSO 

Fire  Support  Officer 

HE 

High  Explosive 

HHC 

Headquarters  and  Headquarters  Company 

HPT 

High  Payoff  Target 

HPTL 

High  Payoff  Target  List 

HUMINT 

Human  Intelligence 

HVT 

High  Value  Target 

IB 

International  Border 

IED 

Improvised  Explosive  Device 

IFV 

Infantry  Fighting  Vehicle 

IMINT 

Imagery  Intelligence 

A-2 


IN  Infantry 

10  Information  Operations 

IPB  Intelligence  Preparation  of  the  Battlefield 

ISR  Intelligence,  Surveillance,  and  Reconnaissance 

JSOTF  Joint  Special  Operations  Task  Force 


LD 

LOC 

LOG  PAC 

LRP 

LT 


Line  of  Departure 
Lines  of  Communication 
Logistics  Package 
Logistics  Release  Point 
Lieutenant 


MABE 

MAJ 

MDMP 

MEDEVAC 

MCOO 

METT-TC 

MI 

MIC 

MICLIC 

MOPP 

MSR  X 

MTOE 


Mission  Analysis  Briefing  Exercise 
Major 

Military  Decision-Making  Process 

Medical  Evacuation 

Modified  Combined  Obstacle  Overlay 

Mission,  Enemy,  Troops,  Terrain  and  Weather,  Timing,  and  Civil 
Factors 

Military  Intelligence 
Mechanized  Infantry  Company 
Mine  Clearing  Line  Charge 
Mission  Oriented  Protective  Posture 
Main  Supply  Route  X 

Modified  Tables  of  Organization  and  Equipment 


N/A 

NAI 

NBC 

NW 


Not  Applicable 
Named  Area  of  Interest 
Nuclear,  Biological,  Chemical 
North  West 


OBJ 

OP 

OPORD 

OPSEC 


Objective 
Observation  Post 
Operations  Order 
Operations  Security 


PAA  Position  Area  for  Artillery 

PIR  Priority  (or  Primary)  Intelligence  Requirement 

PL  Phase  Line 

PLT  Platoon 

POL  Petroleum,  Oils,  and  Lubricants 

PSYOP  Psychological  Operations 


R3  Re-arm,  Re-fuel,  Re-supply 

R3P  Re-arm,  Re-fuel,  Re-supply  Point 


A-3 


REMBASS 

Remotely  Monitored  Battlefield  Sensor  System 

RETRANS 

Retransmission 

ROE 

Rules  of  Engagement 

ROM 

Refuel  on  the  Move 

ROWPU 

Reserve  Osmosis  Water  Purification  Unit 

RSTA 

Reconnaissance,  Surveillance,  and  Target  Acquisition 

SI 

Adjutant/Personnel  Officer 

S2 

Intelligence  Officer 

S3 

Operations  and  Training  Officer 

S4 

Supply/Logistics  Officer 

SA 

Situation  Awareness 

SBCT 

Stryker  Brigade  Combat  Team 

SBF 

Support  by  Fire 

SEE 

Small  Emplacement  Excavator 

IGINT 

Signals  Intelligence 

SIGO 

Signal  Officer 

SITTEMP 

Situation  Template 

SME 

Subject  Matter  Expert 

SOP 

Standing  Operating  Procedure 

SOSO 

Stability  Operations  and  Support  Operations 

SOSR 

Suppress,  Obscure,  Secure,  Reduce 

SP 

Start  Point 

SPF 

Special  Purpose  Forces 

SPOTREP 

Spot  Report 

TAC-P 

Tactical  Air  Control  Party 

TAI 

Targeted  Area  of  Interest 

TARGETS 

Targeted  Acceptable  Responses  to  Generated  Events  or  Tasks 

TF 

Task  Force 

TK 

Tacit  Knowledge 

TLAC 

Think  Like  a  Commander 

TM 

Training  Manual 

TOC 

Tactical  Operations  Center 

TPT 

Tactical  PSYOP  Team 

TRP 

Target  Reference  Point 

TTP 

Tactics,  Techniques,  and  Procedures 

UAV 

Unmanned  Aerial  Vehicle 

UMCP 

Unit  Maintenance  Collection  Point 

VTOC 

Virtual  Tactical  Operations  Center 

XO 

Executive  Officer 
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Appendix  B 

Staff  Roles  Knowledge  Assessment 

BOS  #1  -  COMMAND  AND  CONTROL 

1.  Clearly  identify  the  commander’s  intent  and  vision  of  the  battle. 

Task  Definition:  Clear  identification  of  the  commander’s  intent  and  vision  of  the  battle  requires 
an  in-depth  familiarity  with  the  purpose,  key  tasks,  and  end  state  of  the  mission,  as  well  as  the 
commander’s  priorities  for  all  combat,  combat  support  (CS),  and  combat  service  support  (CSS) 
elements  and  how  he  envisions  their  support  of  his  concept. 

Staff  Officers  Involved:  Executive  Officer  (XO),  Adjutant/Personnel  Officer  (SI),  Intelligence 
Officer  (S2),  Operations  and  Training  Officer  (S3),  Supply/Logistics  Officer  (S4),  Signal  Officer 
(SIGO),  Fire  Support  Officer  (FSO),  Air  Defense  Officer  (ADO),  Engineer  (ENG) 

Rationale:  The  effectiveness  and  timeliness  of  the  war  game  depends  on  clear  identification  of 
the  commander’s  intent  and  vision  of  the  battle.  This  is  because  the  commander  and  his  staff  use 
wargaming  to,  among  other  things,  (a)  determine  how  to  maximize  combat  power  against  the 
enemy  while  protecting  the  friendly  forces  and  minimizing  collateral  damage;  (b)  have  as  near  an 
identical  vision  of  the  battle  as  possible;  and  (c)  determine  the  conditions  and  resources  required 
for  success.  Because  all  staff  officers  must  participate  actively  for  wargaming  to  be  effective,  all 
staff  officers  must  be  able  to  clearly  identify  the  commander’s  intent  and  vision  of  the  battle. 

The  commander  also  plays  a  key  role  in  this  task  by  communicating  his  intent  and  vision 
effectively. 

References:  FM  5-0  (pp.  3-5,  3-33  to  3-34) 

2.  Determine  the  command  and  control  requirements  for  minimizing  the  effect  of 
destruction  of  friendly  Command  Posts  (CPs). 

Task  Definition:  Determination  of  the  command  and  control  (C2)  requirements  for  minimizing 
the  effect  of  destruction  of  friendly  CPs  requires  identification  of  (a)  succession  of  command;  (b) 
provisions  for  redundancy  in  communications  by  having  backup  at  key  locations;  (c)  SOPs  for 
subordinates  to  follow  during  interruptions  in  communications;  and  (d)  alternative  means  of 
communication  if  main  means  are  eliminated. 

Staff  Officers  Involved:  XO,  S3,  SIGO 

Rationale:  The  XO  is  the  task  force’s  “chief  of  staff’  and  second  in  command,  and  must  be 
prepared  to  assume  the  duties  of  commander  at  any  time.  He  is  responsible  for  directly 
supervising  the  main  CP  and  headquarters  cell,  including  displacement,  protection,  security,  and 
communications.  The  S3  makes  recommendations  for  C2  requirements  based  on  his  knowledge 
of  the  flow  of  battle,  how  the  CPs  will  move  with  the  battle,  and  how  the  fight  will  be  controlled. 
The  SIGO  advises  on  all  communications  and  electronics  matters  including  the  positioning  of  C2 
elements.  He  is  the  point  of  contact  for  the  issue  of  signal  operating  instruction  during 
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operations,  as  well  as  for  communications  troubleshooting.  He  is  also  responsible  for  providing 
retransmission  capabilities  to  the  task  force.  He  ensures  that  communication  resources  and 
support  are  adequate  to  meet  mission  requirements. 

References:  Commander’s  Battle  Staff  Handbook  (p.  10,  56, 119  to  120);  FM  3-90.2  (pp.  1 1-7 
to  1-8);  FM  101-5  (p.4-2) 

3.  Determine  or  refine  the  Commander’s  Critical  Information  Requirements  (CCIR). 

Task  Definition:  The  CCIR  identify  information  needed  by  the  commander  to  support  his 
battlefield  visualization  and  to  make  critical  decisions  during  execution.  This  information 
represents  what  a  commander  needs  to  know  in  a  specific  situation  to  make  a  particular  decision 
in  a  timely  manner.  Therefore,  determining  the  CCIR  involves  identifying  requirements  for 
information  and  recognizing  the  importance  of  these  requirements  to  the  commander’s  decision¬ 
making  process  (e.g.,  their  importance  to  confirming  or  denying  a  particular  enemy  COA). 
Refinement  of  the  CCIR  requires  the  identification  of  changes  in  (or  updates  to)  the  situation, 
environment,  or  timeframe  and  their  corresponding  implications  for  what  the  commander  needs 
to  know. 

Staff  Officers  Involved:  XO,  SI,  S2,  S3,  S4,  SIGO,  FSO,  ADO,  ENG 

Rationale:  The  XO  is  responsible  for  managing  the  commander’s  CCIR.  The  S2  is  responsible 
for  coordinating  with  the  entire  staff  and  recommending  PIR  for  the  commander’s  CCIR.  Each 
staff  officer  provides  technical  assistance  to  the  S2  in  their  areas  of  interest,  while  studying  and 
evaluating  the  enemy  capabilities  in  their  areas  of  interest  and  supporting  the  battlefield 
surveillance  plan.  If  assigned  to  the  battalion/task  force,  the  chemical  officer  (CHEMO)  is  an 
important  contributor  to  CCIR  recommendations.  Note  that  staff  officers  nominate  information 
requirements  to  become  CCIR,  but  the  commander  alone  decides  what  information  is  critical. 

References:  FM  5-0  (pp.  3-18  to  3-19);  FM  101-5  (pp.  4-2,  4-5, 4-10  to  4-17, 4-22  to  4-26) 

4.  Determine  critical  events  and  decision  points. 

Task  Definition:  Critical  events  are  those  events  that  directly  influence  mission  accomplishment, 
including  events  that  trigger  significant  actions  or  decisions,  complicated  actions  requiring 
detailed  study,  and  essential  tasks  identified  during  mission  analysis.  Determination  of  critical 
events  therefore  involves  selecting  from  multiple  events  those  events  whose  successful  outcomes 
are  required  for  mission  accomplishment.  Decision  points  are  events  or  locations  on  the 
battlefield  where  tactical  decisions  are  required  during  mission  execution.  They  indicate  when 
and  where  a  decision  must  be  made  to  have  maximum  impact  on  friendly  or  enemy  COAs. 
Determination  of  decision  points  therefore  requires  selecting  from  multiple  events  or  locations 
those  events  or  locations  where  a  decision  is  critical  to  tactical  success. 

Staff  Officers  Involved:  XO,  SI,  S2,  S3,  S4,  SIGO,  FSO,  ADO,  ENG 
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Rationale:  Determination  of  critical  events  and  decision  points  is  addressed  in  the  task  force 
scheme  of  maneuver,  which  is  developed  by  the  S3  in  collaboration  with  the  SI,  S2,  S4,  SIGO, 
FSO,  ADO,  and  ENG  such  that  the  scheme  of  maneuver  is  integrated  with  enemy,  CS,  and  CSS 
capabilities  and  limitations.  The  XO  is  ultimately  responsible  for  integrating  and  synchronizing 
the  warfighting  plans. 

References:  Commander’s  Battle  Staff  Handbook  (p.  56);  FM  5-0  (pp.  3-37  to  3-38);  FM  101-5 
(P-  4-2) 

5.  Determine  command  post  locations  and  composition  to  support  current  and  planned 
tactical  operations. 

Task  Definition:  There  are  three  CPs  involved  in  task  force  command  and  control:  the  main  CP, 
the  combat  trains  CP  (CTCP),  and  the  forward  support  company  CP  (FSC  CP).  Determination 
of  the  CP  locations  requires  consideration  of  CP  survivability,  communications,  and 
accessibility.  Determination  of  CP  composition  requires  consideration  of  how  to  balance  the 
need  for  rapid  displacement  with  C2  effectiveness,  and  the  need  for  24-hour  security  and 
operations. 

Staff  Officers  Involved:  XO,  SI,  S2,  S3,  S4,  SIGO,  ENG 

Rationale:  Key  considerations  for  positioning  the  CPs  include  space  available,  terrain,  weather, 
and  other  environmental  factors,  enemy  factors,  and  operational  factors,  requiring  input  from  the 
S2,  S3,  SIGO,  and  ENG.  The  S3,  in  coordination  with  the  SIGO,  recommends  general  locations 
of  CPs.  He  ensures  that  the  CPs  will  not  be  in  the  way  of  friendly  operations,  deconflicting  the 
terrain  based  on  his  knowledge  of  the  flow  of  battle  and  other  factors.  The  SIGO  ensures  that 
selected  areas  afford  the  most  in  communications  potential  and  the  least  in  potential  enemy 
electronic  warfare  (EW)  interference.  The  S2,  based  on  his  knowledge  of  the  enemy,  assists  in 
identifying  security  considerations.  The  TF  XO  is  responsible  for  supervising  all  staff  activities 
and  functions  within  the  main  CP  and  headquarters  cell,  including  displacement,  protection, 
security,  and  communications,  and  so  is  positioned  to  make  recommendations  about  main  CP 
composition.  Depending  on  the  type  of  organization,  the  S4  or  headquarters  and  headquarters 
company  (HHC)  commander  (CDR)  (or  FSC  CDR)  is  responsible  for  operations,  movement,  and 
security  of  the  combat  trains  command  post  (CTCP).  The  SI  and  S4  work  closely  in  the  CTCP, 
and  so  are  positioned  to  make  recommendations  about  CTCP  composition. 

References:  Commander’s  Battle  Staff  Handbook  (pp.  56, 119, 121);  FM  3-90.2  (pp.  3-8  to  3- 
9, 11-2,  11-4);  FM  101-5  (p.  4-13) 

6.  Assess  the  potential  effect  of  battle  intensity  on  Soldier  and  leader  will  to  fight. 

Task  Definition:  Assessment  of  the  potential  effect  of  battle  intensity  on  Soldier  and  leader  will 
to  fight  requires  determination  of  battle  intensity  via  casualty  estimates  and  via  estimation  of 
personnel  service  support  losses  (e.g.,  postal  services),  both  of  which  affect  unit  morale,  and 
recognizing  their  implications  for  morale. 
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Staff  Officers  Involved:  XO,  SI 


Rationale:  The  S 1  is  responsible  for  reporting  the  status  of  morale  and  esprit  de  corps,  and  any 
significant  influences  on  the  morale  of  units.  The  XO  is  responsible  for  monitoring  the 
discipline,  morale,  and  combat  and  mobilization  readiness  of  the  staff,  which  is  also  affected  by 
battle  intensity,  and  therefore  can  inform  efforts  to  assess  the  potential  effect  of  battle  intensity 
on  Soldier  and  leader  will  to  fight.  If  assigned,  the  chaplain,  surgeon,  and  command  sergeant 
major  are  key  assets  of  the  battalion/task  force  SI  section  who  aid  in  this  assessment. 

References:  Commander’s  Battle  Staff  Handbook  (p.  29  to  36);  FM  101-5  (p.  4-2,  4-10,  4-21,  4- 
30) 

BOS  #2  -  INTELLIGENCE 

7.  Determine  High  Payoff  Targets  (HPTs). 

Task  Definition:  High-payoff  targets  are  those  targets  whose  loss  to  the  enemy  will  contribute  to 
the  success  of  the  friendly  CO  A.  Determining  HPTs  requires  joint  consideration  of  the  enemy 
commander’s  needs  (high  value  targets  [HVTs]  -  to  include  enemy  combat,  CS,  and  CSS 
elements)  and  the  friendly  concept  of  the  operation. 

Staff  Officers  Involved:  XO,  S2,  S3,  S4,  FSO,  ADO,  ENG 

Rationale:  The  S2,  S3,  and  FSO  form  the  core  of  the  targeting  team.  The  S2  develops  the  ISR 
plan/overlay  to  find  high-payoff  targets  and  participates  in  the  targeting  cell  to  help  develop  the 
HPT  list.  The  S3  (or  lethal  effects  cell,  depending  on  the  organization)  plays  a  key  role  in 
selecting  HPTs,  collaborating  with  the  FSO  to  integrate  the  tactical  scheme  of  maneuver  with 
fires.  Normally,  the  XO  oversees  the  routine  activity  and  coordination  of  the  targeting  process. 
As  required,  other  representatives  within  the  tactical  operations  center  may  also  be  members  of 
the  targeting  team,  including  the  S4,  ENG,  and  ADO.  If  assigned,  the  CHEMO  also  participates 
in  the  targeting  cell.  At  the  brigade  level  and  above,  the  air  liaison  officer  (ALO)  participates  in 
the  targeting  cell. 

References:  Commander’s  Battle  Staff  Handbook  (p.  56,  83  to  90);  FM  6-20-10  (Chs.  1  and  2); 
FM  101-5  (p.4-22  to  4-24) 

8.  Determine  optimal  times  and  locations  to  maximize  enemy  casualties  and  force 
destruction. 

Task  Definition:  Determining  optimal  times  and  locations  to  maximize  enemy  casualties  and 
force  destruction  requires  that  information  regarding  (a)  anticipated  enemy  locations,  strengths, 
and  actions;  and  (b)  potential  enemy  mission,  intent,  objectives,  defensive  locations,  use  of  key 
terrain,  avenues  of  approach  and  routes,  engagement  areas,  and  obstacles  be  incorporated  with 
the  friendly  commander’s  intent,  mission,  and  integrated  scheme  of  maneuver. 

Staff  Officers  Involved:  XO,  S2,  S3,  S4,  FSO,  ADO,  ENG 
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Rationale:  Incorporating  threat  analysis  with  the  scheme  of  maneuver  requires  coordination  and 
information  sharing  between  the  S2,  S4,  ENG,  and  S3.  For  maximum  planning  effectiveness, 
the  scheme  of  maneuver  must  be  resourced  and  integrated  with  fire  support,  obstacles,  and  air 
defense.  The  XO  is  ultimately  responsible  for  integrating  and  synchronizing  the  warfighting 
plans.  If  assigned,  the  CHEMO  is  also  an  important  contributor  because  he  plans  and 
recommends  integration  of  smoke  and  obscurants  into  tactical  operations. 

References:  FM  3-90.2  (p.  5-26  to  5-29);  FM  101-5  (p.  4-2, 4-6, 4-10  to  4-15,  4-23  to  4-25) 

9.  Determine  ways  and  means  to  separate  attacking  enemy  echelons. 

Task  Definition:  Determining  the  ways  and  means  to  separate  attacking  enemy  echelons 
requires  that  information  regarding  (a)  anticipated  enemy  COAs;  and  (b)  potential  enemy 
mission,  intent,  objectives,  use  of  key  terrain,  avenues  of  approach  and  routes,  and  engagement 
areas  be  incorporated  with  the  friendly  commander’s  intent,  mission,  and  integrated  scheme  of 
maneuver. 

Staff  Officers  Involved:  XO,  S2,  S3,  S4,  FSO,  ADO,  ENG 

Rationale:  Incorporating  threat  analysis  with  the  scheme  of  maneuver  requires  coordination  and 
information  sharing  between  the  S2,  S4,  ENG,  and  S3.  For  maximum  planning  effectiveness, 
the  scheme  of  maneuver  must  be  resourced  and  integrated  with  fire  support,  obstacles,  and  air 
defense.  The  XO  is  ultimately  responsible  for  integrating  and  synchronizing  the  warfighting 
plans.  If  assigned,  the  CHEMO  is  also  an  important  contributor  because  he  plans  and 
recommends  the  use  of  flame-field  expedients  to  supplement  unit  defense  and  existing 
minefields  and  barriers. 

References:  FM  3-90.2  (p.  5-26  to  5-29);  FM  101-5  (p.  4-2, 4-6, 4-10  to  4-15,  4-23  to  4-25) 

10.  Determine  ways  and  means  to  force  enemy  elements  into  areas  where  the  commander 
wants  them. 

Task  Definition:  Determining  ways  and  means  to  force  enemy  elements  into  areas  where  the 
commander  wants  them  requires  that  information  regarding  (a)  anticipated  enemy  locations, 
strengths,  and  actions;  and  (b)  potential  enemy  mission,  intent,  objectives,  defensive  locations, 
use  of  key  terrain,  avenues  of  approach  and  routes,  engagement  areas,  and  obstacles  be 
incorporated  with  the  friendly  commander’s  intent,  mission,  and  integrated  scheme  of  maneuver. 

Staff  Officers  Involved:  XO,  S2,  S3,  S4,  FSO,  ADO,  ENG 

Rationale:  Incorporating  threat  analysis  with  the  scheme  of  maneuver  primarily  involves 
coordination  and  information  sharing  between  the  S2,  ENG,  and  S3.  For  maximum  planning 
effectiveness,  the  scheme  of  maneuver  must  be  resourced  and  integrated  with  fire  support, 
obstacles,  and  air  defense.  The  XO  is  ultimately  responsible  for  integrating  and  synchronizing 
the  warfighting  plans.  If  assigned,  the  CHEMO  is  also  an  important  contributor  because  he  plans 
and  recommends  integration  of  smoke  and  obscurants  into  tactical  operations. 
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References:  FM  3-90.2  (p.  5-26  to  5-29);  FM  101-5  (p.  4-2, 4-6, 4-10  to  4-15, 4-23  to  4-25) 

11.  Identify  expected  enemy  air  or  helicopter  threats. 


Task  Definition:  The  probability  of  a  task  force  asset  being  targeted  for  enemy  air  must  be 
assessed  if  economical  allocation  of  ADA  resources  is  to  be  achieved.  Identification  of  assets 
requiring  active  air  defense  protection  involves  consideration  of  targeting  information  provided 
by  intelligence  estimates,  past  enemy  attack  methods,  and  enemy  doctrine. 

Staff  Officers  Involved:  S2,  ADO 

Rationale:  The  ADO  should  work  closely  with  the  S2  during  the  intelligence  preparation  on  the 
battlefield  (IPB)  process  and  is  best  suited  to  prepare  and  brief  the  Air  IPB.  This  continued 
coordination  occurs  during  wargaming.  At  the  brigade  level  and  above,  the  ALO  also  assists  in 
this  task  by  supplying  his  experience  and  his  knowledge  of  high-performance  aircraft. 

References:  Commander’s  Battle  Staff  Handbook  (p.  109,  1 12);  FM  3-90.2  (p.  9-50);  FM  101-5 
(p.  4-22, 4-23) 


12.  Determine  optimal  employment  of  intelligence  collection  assets. 

Task  Definition:  The  intelligence  collection  management  process  involves,  in  part,  defining 
what  information  is  required,  determining  the  best  method  to  collect  information,  and  allocating 
assets  to  gather  information.  Determining  the  optimal  employment  of  intelligence  collection 
assets  therefore  requires  an  understanding  of  what  information  should  be  collected  and  how  this 
information  should  be  collected.  From  this  understanding,  information  requirements  can  be 
prioritized  and  intelligence  assets  can  be  assigned  accordingly. 

Staff  Officers  Involved:  XO,  S2,  S3,  S4,  SIGO,  FSO,  ADO,  ENG 

Rationale:  Development  of  the  intelligence,  surveillance,  and  reconnaissance  (ISR)  plan  is  a 
collaborative  effort  between  the  S3  and  S2,  supported  by  the  remainder  of  the  staff.  Tasking  of 
specific  collection  assets  is  determined  based  on  the  following  factors:  (a)  availability;  (b) 
capability;  (c)  vulnerability;  and  (d)  performance  history,  which  requires  consideration  of 
personnel  and  equipment  status,  signal  capability,  threat  capability,  and  environmental 
conditions.  Moreover,  intelligence  resources  include  scouts,  maneuver  companies,  patrols, 
observation  post  (OPs),  fire  integration  support  team  (FISTs),  field  artillery  (FA),  military 
intelligence  (MI),  army  aviation,  close  air  support  (CAS),  air  defense  artillery,  combat  engineers, 
and  various  CSS  units.  For  these  reasons,  staff  officers  involved  in  the  ISR  plan  may  include  (in 
addition  to  the  S2)  the  S3,  S4,  SIGO,  FSO,  ADO,  and  ENG.  The  TF  XO  supervises  overall 
development  and  synchronization  of  the  ISR  plan. 

References:  Commander’s  Battle  Staff  Handbook  (pp.  42  to  45);  FM  3-90.2  (pp.  4-2  to  4-7,  4- 
25) 
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BOS  #3 -MANEUVER 

13.  Define  branches  and  sequels  to  the  maneuver  scheme. 


Task  Definition:  A  branch  is  a  contingency  plan  or  course  of  action  (an  option  built  into  the 
basic  plan  or  course  of  action)  for  changing  the  mission,  disposition,  orientation,  or  direction  of 
movement  of  the  force  to  aid  success  of  the  current  operation,  based  on  anticipated  events, 
opportunities,  or  disruptions  caused  by  enemy  actions.  Sequels  are  operations  that  follow  the 
current  operation.  They  are  future  operations  that  anticipate  the  possible  outcomes — success, 
failure,  or  stalemate  of  the  current  operation.  Defining  branches  and  sequels  to  the  maneuver 
scheme  involves  identifying  the  need  to  change  the  decisive  plan  and  execution  criteria  and 
developing  an  integrated  plan  for  implementing  the  change. 

Staff  Officers  Involved:  XO,  S2,  S3,  S4,  SIGO,  FSO,  ADO,  ENG 

Rationale:  For  maximum  planning  effectiveness,  the  scheme  of  maneuver  must  be  resourced  and 
integrated  with  fire  support,  obstacles,  and  air  defense.  The  XO  is  ultimately  responsible  for 
integrating  and  synchronizing  the  warfighting  plans.  If  assigned,  the  CHEMO  is  also  an 
important  contributor  because  he  plans  and  recommends  integration  of  smoke  and  obscurants 
into  tactical  operations.  At  the  brigade  level  and  above,  the  ALO  also  participates  in  defining 
branches  and  sequels,  coordinating  tactical  air  support  missions  with  fire  support  and  the 
appropriate  airspace  command  and  control  element. 

References:  FM  3-0  (pp.  4-25;  6-5);  FM  3-90.2  (p.  5-26  to  5-29);  FM  101-5  (p.  4-2, 4-6, 4-10  to 
4-15, 4-22  to  4-25) 

14.  Identify  triggers  for  the  initiation  of  direct  and  indirect  fires. 

Task  Definition:  Triggers  are  a  physical  point  on  the  ground  or  an  action  or  an  event.  During 
offensive  operations,  a  trigger  is  often  a  maneuver  action  or  event.  In  the  defense,  a  trigger  is 
more  often  a  physical  spot  on  the  ground.  Trigger  development  requires  (a)  determining  the 
position  on  the  ground  where  you  want  to  impact  on  the  enemy  or  to  silhouette  the  enemy;  (b) 
determine  the  enemy  rate  of  movement;  (c)  determine  the  time  of  flight  of  the  rounds  from  the 
weapon  system  firing  the  mission;  (d)  determine  the  processing  time;  (e)  determine  the  total 
mission  time;  and  (f)  place  the  trigger  point  the  required  distance  from  a  planned  target  location 
based  on  total  mission  time  x  speed  of  enemy. 

Staff  Officers  Involved:  S2,  S3,  S4,  FSO,  SIGO,  ENG 

Rationale:  Each  time  the  commander  and  the  S3  discuss  current  or  future  plans,  concepts,  or 
courses  of  action,  the  FSO  participates.  The  FSO  develops  the  fire  support  tasks, 
responsibilities,  and  requirements.  The  S2  participates  in  the  targeting  cell  to  help  develop  the 
HPT  list  and  to  develop  the  observation  plan  and  the  fire  support  execution  matrix.  The  ENG 
assists  with  direct/indirect  fire  integration  with  obstacles,  priorities,  and  obstacle  resourcing. 

Fire  support  planners  must  formulate  tactical  plans  to  reflect  logistics  limitations  and  to  exploit 
logistics  capabilities.  Ammunition,  fuel,  food,  water,  maintenance,  transportation,  and  medical 
support  are  all  critical  to  sustaining  fire  support  operations.  Logistics  sustainment  is  a  central, 
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potentially  decisive  aspect  of  operations,  not  an  adjunct  to  them.  The  SIGO  advises  on  all 
communications  and  electronics  matters  and  is  responsible  for  providing  retransmission 
capabilities  to  the  task  force.  He  ensures  that  communication  resources  and  support  are  adequate 
to  meet  mission  requirements.  If  assigned,  the  CHEMO  participates  in  the  targeting  cell.  At  the 
brigade  level  and  above,  the  ALO  participates  in  identifying  triggers,  helping  to  plan  the 
simultaneous  employment  of  air  and  surface  fires. 

References:  Commander’s  Battle  Staff  Handbook  (pp.  83,  98);  FM  3-90.2  (Appendix  G);  FM 
101-5  (pp.  4-22  to  4-24) 

15,  Determine  route  prioritization  for  movement. 

Task  Definition:  The  objective  of  a  successful  move  is  for  the  unit  to  arrive  at  its  destination  in  a 
condition  suitable  to  its  probable  employment.  The  goal  of  all  movement  planning  is  to  retain 
flexibility  to  execute  a  variety  of  plans  to  meet  ever-changing  conditions.  Movement  planning 
involves  determination  of  the  destination,  routes,  orders  of  march,  rates  of  march,  times  that  each 
serial  or  march  element  will  arrive  and  clear  its  start  point  (SP),  intervals,  speeds,  scheduled 
maintenance  halts,  communications,  and  location  of  the  commander.  An  effective  movement 
order  requires  the  best  available  information  on  the  enemy,  terrain,  weather,  unit  capabilities,  and 
civil  considerations. 

Staff  Officers  Involved:  S2,  S3,  S4,  ADO,  ENG 

Rationale:  The  integration  of  and  support  from  combat  and  CS,  such  as  artillery,  air  defense, 
intelligence,  military  police,  and  engineers,  are  critical  for  a  successful  tactical  movement.  The 
S3  section  develops  the  detailed  movement  order,  with  the  assistance  of  the  S4  section,  in 
accordance  with  the  commander’s  established  priorities.  In  coordination  with  the  ENG,  the  S4 
ensures  that  routes  are  adequate  to  support  the  movement  of  the  types  and  numbers  of  vehicles 
and  supplies  projected  for  movement. 

References:  FM  3-90  (p.  14-12  to  14-13) 

16.  Define  the  task  organization  requirements. 

Task  Definition:  Task  organization  is  the  process  of  allocating  available  assets  to  subordinate 
commanders  and  establishing  their  command  and  support  relationships.  Determining  task 
organization  requirements  during  the  war  game  involves  identifying  what  combat  power  is 
needed,  where,  when,  and  how  frequently  it  will  be  needed.  Successful  task  organization 
requires  understanding  (a)  the  mission,  including  the  higher  commander’s  intent  and  concept  of 
operations;  (b)  Army  doctrinal  tenets  and  tactics;  (c)  the  battlefield  framework;  (d)  the  roles  and 
interrelations  of  operating  systems;  (e)  the  status  of  available  forces,  including  morale,  training, 
and  capabilities  of  equipment;  (f)  specific  unit  capabilities,  limitations,  strengths,  and 
weaknesses;  (g)  the  risks  inherent  to  the  plan;  and  (h)  subordinate  commander’s  abilities, 
especially  the  ability  to  apply  combined  arms  doctrine. 

Staff  Officers  Involved:  XO,  SI,  S2,  S3,  S4,  SIGO,  FSO,  ADO,  ENG 
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Rationale:  Every  staff  officer  is  responsible  for  recommending  the  organization  for  combat, 
allocations  to  subordinate  units,  and  command  and  support  relationships  between  subordinate 
units  and  organic  units  in  their  area  of  interest.  The  XO  ensures  information  flow  between  the 
staff  and  commander  on  staff  recommendations  and  the  commander’s  decisions. 

References:  Commander’s  Battle  Staff  Handbook  (p.  10);  FM  101-5  (p.  4-4,  Appendix  F) 

17.  Define  force  protection  criteria. 

Task  Definition:  Force  protection  consists  of  those  actions  taken  to  prevent  or  mitigate  hostile 
actions  against  personnel  (to  include  family  members),  resources,  facilities,  and  critical 
information.  Defining  force  protection  criteria  requires  the  identification  of  threats  and  their 
associated  hazards,  determining  the  risk  associated  with  each  hazard,  and  balancing  resource 
constraints  against  the  risk.  Resources  allocated  to  force  protection  may  be  devoted  to 
aggressive  counterintelligence  and  threat  assessments,  operations  security  (OPSEC),  troop 
dispersion,  camouflage,  local  security,  field  fortifications,  protection  of  electronic  links  and 
nodes  (including  combat  troops  with  electronic  devices),  and  army  air  and  missile  defense. 

Staff  Officers  Involved:  XO,  SI,  S2,  S3,  S4,  SIGO,  FSO,  ADO,  ENG 

Rationale:  The  XO  is  responsible  for  supervising  integration  of  risk  management  across  the 
staff.  Each  staff  officer  helps  the  commander  eliminate  unnecessary  risks,  in  part,  by  (a) 
analyzing  his  functional  area  and  applying  risk  management  during  the  MDMP;  (b)  identifying 
constraints  in  the  higher  commander’s  risk  guidance;  (c)  including  risks  and  recommending  ways 
to  reduce  their  impact  in  the  staff  estimate;  (d)  determining  the  effectiveness  of  hazard/risk 
controls  and  continuously  assessing  their  suitability,  feasibility,  and  acceptability;  and  (e) 
continuously  identifying  hazards,  assessing  initial  and  residual  risks  for  each  hazard, 
recommending  control  measures  to  reduce  the  risk  to  the  force.  The  HHC  (or  FSC)  CDR  and  the 
surgeon  may  help  in  identifying  risk.  If  assigned,  the  CHEMO  is  a  critical  asset  in  assessing  risk 
to  the  unit  from  enemy  nuclear,  biological,  and/or  chemical  attacks. 

References:  FM  3-0  (p.  4-8  to  4-9);  FM  100-14  (pp.  1-6,  3-6);  FM  101-5  (pp.  4-7, 4-23, 4-24) 

18.  Determine  effect  of  limited  visibility  on  combat,  combat  support  (CS),  and  combat 
service  support  (CSS)  operations. 

Task  Definition:  Commanders  plan  for  the  effects  of  adverse  or  limited  visibility  on  weapons 
systems  and  optical  and  thermal  devices.  A  plan  that  succeeds  in  clear  conditions  may  be  less 
effective  during  bad  weather.  Branches  to  the  basic  plan  should  address  necessary  modifications 
during  periods  of  reduced  visibility.  Defining  branches  to  the  basic  plan  to  address  limited 
visibility  involves  changing  execution  criteria  to  reflect  limited-visibility  effects  on  weapons 
systems  and  optical  and  thermal  devices,  and  developing  an  integrated  plan  for  implementing  the 
change. 

Staff  Officers  Involved:  XO,  S2,  S3,  S4,  FSO,  ADO,  ENG 
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Rationale:  For  maximum  planning  effectiveness,  the  scheme  of  maneuver  must  be  resourced  and 
integrated  with  fire  support,  obstacles,  and  air  defense.  The  XO  is  ultimately  responsible  for 
integrating  and  synchronizing  the  warfighting  plans.  The  HHC  (or  FSC)  CDR  also  aid  in 
determining  the  effect  of  limited  visibility  on  CSS  operations. 

References:  FM  3-0  (p.  8-14);  FM  3-90.2  (p.  5-26  to  5-29);  FM  101-5  (p.  4-2,  4-6,  4-10  to  4-15, 
4-25) 

BOS  #4  -  FIRE  SUPPORT 

19.  Integrate  fire  support  with  maneuver  and  priorities. 

Task  Definition:  Integration  of  fire  support  with  maneuver  and  priorities  requires  that  fire 
support  planning  is  performed  concurrently  with  the  development  of  the  scheme  of  maneuver. 

Staff  Officers  Involved:  S3,  S4,  FSO 

Rationale:  Each  time  the  commander  and  the  S3  discuss  current  or  future  plans,  concepts,  or 
courses  of  action,  the  FSO  participates.  The  FSO  develops  the  fire  support  tasks, 
responsibilities,  and  requirements.  The  S3  recommends  integrated  schemes  of  tactical  maneuver 
and/or  dispositions  and  fires,  including  nuclear  and  chemical  fires.  Fire  support  planners  must 
formulate  tactical  plans  to  reflect  logistics  limitations  and  to  exploit  logistics  capabilities. 
Ammunition,  fuel,  food,  water,  maintenance,  transportation,  and  medical  support  are  all  critical 
to  sustaining  fire  support  operations.  Logistics  sustainment  is  a  central,  potentially  decisive 
aspect  of  operations,  not  an  adjunct  to  them.  If  assigned,  the  ALO  may  also  contribute  to  this 
task. 

References:  Commander’s  Battle  Staff  Handbook  (pp.  56,  83  to  84);  FM  6-20  (Ch.  3) 

20.  Update  High  Payoff  Target  List  (HPTL). 

Task  Definition:  The  HPTL  is  a  prioritized  list  of  high-payoff  targets  (HPTs).  The  HPTs  are 
those  targets  whose  loss  to  the  enemy  will  contribute  to  the  success  of  the  friendly  CO  A. 

Updating  the  HPTL  requires  joint  consideration  of  updates  regarding  the  enemy  commander’s 
needs  (HVTs  -  to  include  enemy  combat,  CS,  and  CSS  elements)  and  updates  regarding  the 
friendly  concept  of  the  operation. 

Staff  Officers  Involved:  XO,  S2,  S3,  FSO,  ADO,  ENG 

Rationale:  The  S2,  S3,  and  FSO  form  the  core  of  the  targeting  team.  The  S2  develops  the  ISR 
plan/overlay  to  find  high-payoff  targets  and  participates  in  the  targeting  cell  to  help  develop  the 
HPT  list.  Normally,  the  XO  oversees  the  routine  activity  and  coordination  of  the  targeting 
process.  As  required,  other  representatives  within  the  TOC  may  also  be  members  of  the 
targeting  team,  including  the  ENG,  and  ADO.  If  assigned,  the  CHEMO  participates  in  the 
targeting  meeting.  At  the  brigade  level  and  above,  the  ALO  also  participates  in  the  targeting 
meeting. 
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References:  Commander’s  Battle  Staff  Handbook  (p.  83  to  90);  FM  6-20-10  (Chs.  1  and  2);  FM 
101-5  (pp.  4-22  to  4-24) 

21.  Synchronize  lethal  and  nonlethal  fires  to  support  task  force  Intelligence,  Surveillance, 
and  Reconnaissance  (ISR)  operations. 

Task  Definition:  The  desired  effects  of  fires  are  to  (a)  suppress  enemy  forces  engaging  task  force 
ISR  forces;  (b)  protect  the  movement  and  infiltration  of  ISR  forces  and  target  acquisition  assets; 
(c)  inflict  casualties  and  force  enemy  units  to  deploy  early  and  reveal  his  main  attack;  (d)  reduce 
the  enemy’s  capability  to  gain  information  by  eliminating  enemy  reconnaissance  forces  and 
intelligence-gathering  assets;  (e)  deceive  the  enemy  as  to  the  location  of  task  force  units;  (f)  slow 
and  canalize  enemy  movement  to  provide  better  targets  for  maneuver  direct  fire  systems;  and  (g) 
suppress  enemy  direct  and  indirect  fire  weapons. 

Staff  Officers  Involved:  S2,  S3,  S4,  FSO 

Rationale:  The  FSO  develops/synchronizes  the  fire  support  tasks,  responsibilities,  and 
requirements.  The  S3  has  responsibility  for  developing  the  ISR  plan  with  the  FSO  and  S2.  Fire 
support  planners  must  formulate  tactical  plans  to  reflect  logistics  limitations  and  to  exploit 
logistics  capabilities.  Ammunition,  fuel,  food,  water,  maintenance,  transportation,  and  medical 
support  are  all  critical  to  sustaining  fire  support  operations.  Logistics  sustainment  is  a  central, 
potentially  decisive  aspect  of  operations,  not  an  adjunct  to  them.  At  the  division  level  and  above, 
the  electronic  warfare  officer  also  participates  in  this  task. 

References:  Commander’s  Battle  Staff  Handbook  (pp.  59,  84);  FM  6-20-40;  FM  6-20-50;  FM 
101-5  (p.  4-25) 

22.  Synchronize  lethal  and  non-lethal  fires  to  support  task  force  offensive  operations. 

Task  Definition:  Fires  in  support  of  offensive  operations  are  to  be  planned  for  (1)  terrain  that 
will  be  traversed  by  task  force  units;  (2)  protecting  task  force  flanks;  and  (3)  achieving  task  force 
objectives.  For  fires  planned  for  terrain  that  will  be  traversed  by  task  force  units,  the  desired 
effects  are  to  (a)  suppress,  neutralize,  or  destroy  enemy  OPs  and  target  acquisition  systems;  (b) 
suppress,  neutralize,  or  destroy  enemy  direct  fire  systems;  (c)  deny  enemy  attack  helicopters  use 
of  potential  attack  positions;  and  (d)  suppress,  neutralize,  or  destroy  enemy  elements  that  are  to 
be  bypassed.  For  fires  planned  to  protect  TF  flanks,  the  desired  effects  are  to  protect  TF  units 
during  movement  and  reduce  the  number  of  TF  units  committed  to  flank  security.  For  fires 
planned  on  TF  objectives,  the  desired  effects  are  to  (a)  concentrate  fires  to  suppress,  neutralize, 
and  destroy  forward  enemy  elements;  (b)  create  points  of  penetration  into  enemy  defenses;  (c) 
suppress  enemy  forces  which  will  respond  to  penetrations  or  breakthrough  of  enemy  positions; 
and  (d)  assist  TF  forces  to  maneuver  to  positions  to  provide  direct  fires  against  enemy  forces  in 
the  defense. 

Staff  Officers  Involved:  S3,  S4,  FSO 
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Rationale:  Each  time  the  commander  and  the  S3  discuss  current  or  future  plans,  concepts,  or 
courses  of  action,  the  FSO  participates.  The  FSO  develops  the  fire  support  tasks, 
responsibilities,  and  requirements.  The  S3  recommends  integrated  schemes  of  tactical  maneuver 
and/or  dispositions  and  fires,  including  nuclear  and  chemical  fires.  Fire  support  planners  must 
formulate  tactical  plans  to  reflect  logistics  limitations  and  to  exploit  logistics  capabilities. 
Ammunition,  fuel,  food,  water,  maintenance,  transportation,  and  medical  support  are  all  critical 
to  sustaining  fire  support  operations.  Logistics  sustainment  is  a  central,  potentially  decisive 
aspect  of  operations,  not  an  adjunct  to  them.  If  assigned,  the  CHEMO  is  an  important 
contributor  because  he  plans  and  recommends  integration  of  smoke  and  obscurants  into  tactical 
operations.  At  the  division  level  and  above,  the  electronic  warfare  officer  and  the  psychological 
operations  officer  would  also  participate  in  this  task. 

References:  Commander’s  Battle  Staff  Handbook  (pp.  56,  83  to  84);  FM  6-20  (Ch.  3);  FM  6- 
20-40;  FM  6-20-50;  FM  101-5  (pp.  4-25, 4-27,  4-28) 

23.  Develop  an  observation  plan  that  assigns  (a)  responsibilities  to  target  acquisition 
systems;  and  (b)  observers  for  the  employment  of  indirect  fires  against  designated  targets 
and  determination  of  damage  assessments. 

Task  Definition:  The  observation  plan  should  address  where  observers  need  to  be,  security, 
communications,  how  the  observer  gets  into  position,  what  the  observer  is  to  accomplish,  and 
disengagement  criteria  if  necessary.  The  steps  in  developing  an  observation  plan  are  (a)  identify 
the  requirements  for  an  observation  post  (OP);  (b)  conduct  terrain  analysis  to  determine  possible 
OP  locations;  (c)  allocate  the  asset;  (d)  select  the  OP  from  among  the  possibilities;  (e)  plan  for 
insertion  and  occupation  of  the  OP;  (f)  coordinate  the  passage  of  the  OP  through  friendly  forces, 
if  required;  and  (g)  plan  indirect  fires,  electronic  warfare  support,  medical  support,  extraction, 
security,  and  re-supply  to  support  the  insertion  and  occupation  plan. 

Staff  Officers  Involved:  S2,  S3,  SIGO,  FSO 

Rationale:  The  observation  plan  is  developed  collaboratively  among  the  FSO,  S2,  and  S3.  The 
S3  develops  a  scheme  that  will  optimize  observation  and  fields  of  fire,  based  on  terrain,  visibility 
conditions,  and  weapon  system  capabilities  (for  both  friendly  and  enemy  forces).  The  SIGO 
ensures  selected  areas  afford  the  most  in  communications  potential  and  the  least  in  potential 
enemy  EW  interference.  At  the  brigade  level,  the  scout  platoon  leader  should  also  participate  in 
this  task. 

References:  Commander’s  Battle  Staff  Handbook  (p.  28,  59,  65,  88, 121);  FM  3-90.2  (p.  9-10  to 
9-11);  FM  6-20  (Ch.  3) 

24.  Verify  sensor  taskings  to  provide  targetable  intelligence  in  a  timely  manner  for  high 
payoff  targets  (HPTs). 

Task  Definition:  Tasking  the  right  sensor  for  a  collection  task  at  the  right  time  is  a  critical 
function  in  the  targeting  process.  Clear  and  concise  taskings  must  be  given  to  each  agency 
controlling  sensors  within  the  force  or  unit.  Effective  sensor  tasking  requires  that  staffs  choose 
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small  areas  selectively  on  the  basis  of  analysis  of  the  IPB  product  most  likely  to  produce  the 
desired  targets. 

Staff  Officers  Involved:  S2,  S3,  FSO 

Rationale:  Constant  coordination  is  required  among  operations,  intelligence,  and  fire  support 
staff  sections  to  ensure  the  effective  employment  of  surveillance  and  target  acquisition  resources. 

References:  FM  6-20  (Ch.  2) 

BOS  #5  -  MOBILITY/SURVIVABILITY 

25.  Assess  potential  enemy  actions  against  the  task  force’s  efforts  to  bypass  or  overcome 
obstacles. 

Task  Definition:  The  assessment  of  potential  enemy  actions  against  TF  efforts  to  bypass  or 
overcome  obstacles  requires  consideration  of  (a)  the  implications  of  enemy  mission  and  intent 
for  enemy  obstacle  placement  and  defensive  action;  (b)  estimated  enemy  intelligence  regarding 
friendly  mission,  intent,  and  capability;  (c)  estimated  enemy  capability  and  combat  power 
relative  to  friendly  capability  and  combat  power;  and  (d)  terrain  and  weather  constraints  on  the 
enemy’s  options  for  responding  to  TF  efforts. 

Staff  Officers  Involved:  S2,  S3,  FSO,  ENG 

Rationale:  The  ENG,  along  with  the  S2,  combines  the  doctrinal  enemy  template,  the  terrain 
analysis,  and  the  other  battlefield  effects  to  gain  an  appreciation  of  how  the  enemy  will  use  the 
terrain  to  fight.  The  S3  uses  intelligence  provided  by  the  S2  to  analyze  the  enemy’s  most  recent 
activities  and  intentions  and  evaluate  possible  enemy  COAs.  Each  time  the  commander  and  the 
S3  discuss  current  or  future  plans,  concepts,  or  courses  of  action,  the  FSO  participates. 

References:  Commander’s  Battle  Staff  Handbook  (pp.  59,  83);  FM  5-71-2  (Appendix  A) 

26.  Define  reconnaissance  requirements  to  identify  points  of  penetration  into  enemy 
obstacles  and  river  crossing  sites. 

Task  Definition:  Defining  the  reconnaissance  requirements  to  identify  points  of  penetration 
involves  identifying  gaps  between  what  is  known  about  how  the  enemy  will  employ  obstacles, 
fortifications,  and  mobility  assets  during  his  defense  and  what  needs  to  be  known  in  order  to 
support  timely,  critical  maneuver  decisions  that  must  be  made  during  the  offensive  operation. 

Staff  Officers  Involved:  S2,  S3,  S4,  FSO,  ENG 

Rationale:  The  ENG  identifies  obstacle  intelligence  and  nominates  priority  intelligence 
requirements  for  inclusion  into  the  CCIR.  The  engineer,  along  with  the  S2,  combines  the 
doctrinal  enemy  template,  the  terrain  analysis,  and  the  other  battlefield  effects  to  gain  an 
appreciation  of  how  the  enemy  will  use  the  terrain  to  fight.  The  ENG  works  with  the  S3,  S4,  and 
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FSO  to  develop  the  engineer  plan,  to  provide  resources  to  support  the  plan,  and  to  coordinate 
fires  with  breaching  and  obstacles. 

References:  FM  5-71-2  (Ch.  3,  Appendix  A) 

27.  Integrate  engineers  into  maneuver  formations  to  maintain  momentum,  with  the  bulk  of 
mobility  assets  with  the  breach  force. 

Task  Definition:  A  highly  mobile  engineer  force,  well  forward  and  integrated  into  maneuver 
formations  is  critical  to  maintaining  the  momentum  of  the  attack.  Engineers  assist  in 
maintaining  momentum  by  supporting  combined  arms  breaching,  land  handover,  forward 
passage  of  follow-on  forces,  and  clearing  and  gap  crossing.  Integration  of  engineers  into 
maneuver  formations  requires  that  specific  arrangements  be  made  for  handing  over  obstacles 
from  forward  breaching  units  to  engineers  for  lane  improvement  and  obstacle  clearance.  The 
amount  and  type  of  engineer  equipment  needed  in  the  offense  must  also  be  considered. 

Staff  Officers  Involved:  S3,  S4,  FSO,  ENG 

Rationale:  The  ENG  works  with  the  S3,  S4,  and  FSO  to  develop  the  engineer  plan,  to  provide 
resources  to  support  the  plan,  and  to  coordinate  fires  with  breaching  and  obstacles. 

References:  Commander’s  Battle  Staff  Handbook  (pp.  98,  100  to  101);  FM  5-71-2  (Ch.  3) 

28.  Determine  Family  of  Scatterable  Mines  (FASCAM)  employment. 

Task  Definition:  Scatterable  mine  systems  enable  tactical  commanders  to  emplace  minefields  in 
enemy  held  terrain,  contaminated  territory,  or  in  other  areas  where  it  is  not  possible  to  emplace 
conventional  minefields.  The  FASCAM  is  designed  to  be  delivered  or  dispensed  remotely  by 
aircraft,  artillery,  or  by  ground  dispenser,  and  may  be  delivered  by  itself  or  in  conjunction  with 
other  munitions.  Planning  the  employment  FASCAM  minefields  requires  consideration  of 
delivery  error,  availability  of  tubes,  competing  demands  for  field  artillery,  and  the  duration  of  the 
mines. 

Staff  Officers  Involved:  S2,  S3,  FSO,  ENG 

Rationale:  The  effective  employment  of  FASCAM  assets  requires  careful  coordination  among 
the  ENG,  FSO,  S3,  and  S2.  The  ENG  is  responsible  for  providing  expertise  on  the  employment 
of  all  types  of  FASCAM.  He  determines  location,  size,  time  and  density  of  the  minefields.  He 
coordinates  with  the  S3  and  FSO  to  ensure  systems  are  available  at  the  time  and  location  for 
placement.  The  FSO  provides  the  technical  expertise  to  the  ENG  concerning  the  employment  of 
field  artillery  (FA)-delivered  FASCAM.  Normally,  the  FSO  obtains  the  safety  zone  (size)  of  the 
minefield.  Because  FASCAM  can  be  delivered  by  air,  the  ALO  (at  the  battalion  level  or  above) 
or  the  S3  Air  (if  assigned)  should  also  participate  in  determining  FASCAM  employment. 

References:  Commander’s  Battle  Staff  Handbook  (pp.  60,  96);  FM  3-90.2  (p.  9-43);  FM  6-20-30 
(Appendix  C) 


B-14 


29.  Define  emplacement  criteria  for  obstacles  and  mines  to  protect  the  task  force  flanks 
and  block  enemy  counterattacks. 

Task  Definition:  Obstacles  must  support  present  and  future  tactical  plans,  be  logistically 
supportable,  and  fully  coordinated.  Some  important  factors  to  be  considered  when  defining 
emplacement  criteria  include  (a)  type  of  mission;  (b)  type  of  obstacle;  (c)  requirements  of  future 
plans;  (d)  enemy  strengths  and  weaknesses;  (e)  terrain  and  weather;  (f)  available  time,  materials, 
manpower,  and  equipment;  (g)  and  effects  on  the  local  population. 

Staff  Officers  Involved:  S2,  S3,  S4,  FSO,  ADO,  ENG 

Rationale:  These  elements  of  the  combined  arms  battalion/task  force  must  be  involved  in  the 
obstacle  planning  and  employment  process  in  order  to  extract  the  greatest  cost  from  the  enemy. 
Obstacles  must  be  resourced,  and  integrated  with  the  scheme  of  maneuver,  fire  support,  and  air 
defense.  If  assigned,  the  CHEMO  is  also  an  important  contributor  because  he  plans  and 
recommends  the  use  of  flame-field  expedients  to  supplement  unit  defense  and  existing 
minefields  and  barriers. 

References:  Commander’s  Battle  Staff  Handbook  (p.  85);  FM  5-103  (Ch.  4);  FM  101-5  (pp.  4- 
23,4-24) 

30.  Determine  requirements  and  priorities  for  force  protection,  to  include  survivability 
positions  for  vehicles,  weapons,  systems,  and  equipment. 

Task  Definition:  The  determination  of  requirements  for  force  protection  includes  consideration 
of  each  asset’s  (a)  exposure  to  direct,  indirect,  and  tactical  air  fire;  (b)  vulnerability  to  discovery 
and  location;  (c)  capability  to  move  to  avoid  detection,  or  to  displace  before  counterfire  arrives; 
(d)  armor  suitability  to  cover  direct  small  caliber  fire,  indirect  artillery  and  mortar  fire,  and  direct 
fire  antitank  weapons;  (e)  distance  from  the  forward  line  of  own  troops  (FLOT)  which  affects  the 
likelihood  of  acquisition  as  a  target,  vulnerability  to  artillery  and  air  bombardment,  and  chance  of 
direct  contact  with  the  enemy;  (f)  availability  of  natural  cover;  (g)  any  unique  equipment  item, 
the  loss  of  which  would  make  other  equipment  worthless;  and  (h)  ability  to  establish  positions 
with  organic  equipment.  The  enemy's  engagement  priority,  including  which  forces  the  threat 
most  likely  will  engage  first,  should  also  be  considered.  Based  on  a  vulnerability  analysis  of 
systems  that  need  protecting  in  the  tactical  situation,  the  maneuver  commander  develops  the 
priorities  for  protective  activities.  Setting  survivability  priorities  is  a  maneuver  commander's 
decision  based  on  the  engineer's  advice. 

Staff  Officers  Involved:  XO,  S2,  S3,  S4,  FSO,  ENG 

Rationale:  Planning  survivability  missions  requires  staff  input  on  the  following  considerations: 
military  intelligence  (enemy  activity,  terrain,  weather,  and  weapon  types),  operations  (tactical 
maneuver,  fire  support,  and  engineer  support),  and  administration/logistics.  The  XO  is 
ultimately  responsible  for  integrating  and  synchronizing  the  warfighting  plans. 

References:  FM  5-103  (Ch.  2) 
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BOS  #6  -  AIR  DEFENSE 

31.  Determine  air  defense  support  and  priorities. 

Task  Definition:  Air  defense  priorities  are  established  to  ensure  effective  and  continuous  support 
of  offensive  and  defensive  operations.  Determining  air  defense  support  involves  determining  for 
each  air  defense  asset  (a)  criticality  -  the  degree  to  which  the  asset  is  essential  to  mission 
accomplishment;  (b)  vulnerability  -  the  degree  to  which  an  asset  is  susceptible  to  attack  or  to 
damage  if  attacked;  (c)  recuperability  -  the  degree  to  which  the  asset  can  recover  from  inflicted 
damage  in  terms  of  time,  equipment,  and  available  manpower  to  again  perform  its  mission;  and 
(d)  threat  -  the  probability  of  an  asset  being  targeted  for  attack  by  enemy  air  must  be  assessed  if 
economical  allocation  of  air  defense  officer  (ADA)  resources  is  to  be  achieved.  The  criticality, 
vulnerability,  recuperability,  and  threat  of  each  asset  must  be  weighed  against  its  total 
contribution  to  the  battle.  Priorities  for  protection  may  include  maneuver  elements,  fire  support, 
engineer  elements,  C2  nodes,  and  logistics  assets.  The  air  defense  plan  must  support  the 
commander’s  scheme  of  maneuver. 

Staff  Officers  Involved:  XO,  S2,  S3,  S4,  FSO,  ADO 

Rationale:  The  ADO  works  with  the  S2  to  determine  air  threat,  and  assists  the  S3  in  planning  the 
air  defense  portion  of  the  operation.  Tactical-level  air  and  missile  defense  is  primarily  the 
responsibility  of  ADA,  but  maneuver,  fire  support,  aviation,  and  intelligence  elements  must 
participate  directly.  Logistics  provides  the  means  for  all  air  and  missile  defense  operations. 

Each  participant  has  a  specific  role  in  tactical  air  and  missile  defense  plans  and  operations. 

These  integrated  roles  are  mutually  supporting.  The  ADO  must  work  closely  with  the  S3  to 
determine  air  defense  asset  allocation,  positioning,  and  missions  in  accordance  with  the  priorities 
established  by  the  commander.  In  addition,  the  ADO  coordinates  with  the  S3  Air  (if  assigned), 
FSO,  and  forward  air  controller  for  the  appropriate  air  defense  posture  and  Army  airspace  C2. 

The  XO  is  ultimately  responsible  for  integrating  and  synchronizing  the  warfighting  plans. 

References:  Commander’s  Battle  Staff  Handbook  (pp.  60, 112);  FM  3-90.2  (p.  9-50  to  9-51) 

32.  Define  early  warning  requirements. 

Task  Definition:  Early  warning  of  enemy  air  attack  is  a  passive  air  defense  measure.  It  is  vital  if 
the  principles  of  early  engagement  and  defense  in  depth  are  to  be  achieved.  Defining  early 
warning  requirements  involves  joint  consideration  of  the  possible  air  threat  and  the  capability  of 
available  air  defense  assets. 

Staff  Officers  Involved:  S2,  S3,  ADO 

Rationale:  The  ADO  works  with  the  S2  to  determine  air  threat.  The  ADO  evaluates  and 
recommends  passive  measures  for  incorporation  into  the  maneuver  commander's  plans  and 
SOPs,  including  early  warning  systems.  He  advises  the  commander  and  staff  on  the  impact  of 
early  warning  on  air  defense  operations  and  plans  early  warning  operations  within  air  defense. 

The  ALO  (at  the  brigade  level  or  above)  or  the  S3  Air  (if  assigned)  contribute  to  this  task. 
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References:  FM  44-100  (Chs.  3  and  6);  FM  101-5  (p.  4-22, 4-23) 

33.  Determine  air  defense  decision  points. 

Task  Definition:  Decision  points  (DPs)  are  events  or  locations  on  the  battlefield  where  tactical 
decisions  are  required  during  mission  execution.  They  indicate  when  and  where  a  decision  must 
be  made  to  have  maximum  impact  on  friendly  or  enemy  COAs.  Determination  of  decision 
points  therefore  requires  selecting  from  multiple  events  or  locations  those  events  or  locations 
where  a  decision  is  critical  to  tactical  success.  Air  DPs  are  determined  in  the  same  manner  as  for 
ground  operations.  However,  due  to  the  high  speeds  of  air  systems,  DPs  must  be  placed 
significantly  farther  in  advance  of  the  targeted  area  of  interest  (TAI). 

Staff  Officers  Involved:  XO,  S2,  S3,  FSO,  ADO,  ENG 

Rationale:  Tactical-level  air  and  missile  defense  is  primarily  the  responsibility  of  ADA,  but 
maneuver,  fire  support,  aviation,  and  intelligence  elements  must  participate  directly.  The  ADO 
must  work  closely  with  the  S3  to  determine  air  defense  asset  allocation,  positioning,  and 
missions  in  accordance  with  the  priorities  established  by  the  commander.  In  addition,  the  ADO 
coordinates  with  the  S3  Air  (if  assigned)  or  ALO  (at  the  brigade  level  and  above),  FSO,  and 
forward  air  controller  for  the  appropriate  air  defense  posture  and  Army  airspace  C2.  The  XO  is 
ultimately  responsible  for  integrating  and  synchronizing  the  warfighting  plans. 

References:  Commander’s  Battle  Staff  Handbook  (pp.  56, 60);  FM  5-0  (p.  3-38);  FM  44-100 
(Ch.  6,  Appendix  A);  FM  101-5  (p.  4-2, 4-6, 4-22, 4-23) 

34.  Determine  air  defense  movements  in  support  of  task  force  operations. 

Task  Definition:  Tactical-level  ADA  units  must  have  mobility  equal  to  the  mobility  of  the 
supported  force.  Movement  ensures  that  tactical -level  ADA  forces  can  project  their  operation 
into  any  area  required  by  the  maneuver  force  or  indicated  by  the  threat.  The  ability  to  move  also 
signifies  that  ADA  weapons  systems  are  not  tied  to  a  static  support  base.  The  first  priority  for 
mobility  should  be  planning  moves  that  support  accomplishment  of  the  mission.  Tactical 
situations  may  dictate  additional  moves  to  enhance  survivability. 

Staff  Officers  Involved:  S3,  ADO 

Rationale:  The  ADO  must  work  closely  with  the  S3  to  determine  air  defense  asset  allocation, 
positioning,  and  missions  in  accordance  with  the  priorities  established  by  the  commander.  At  the 
brigade  level,  the  S4  and  FSO  may  also  be  involved  in  determining  air  defense  movements. 

References:  Commander’s  Battle  Staff  Handbook  (p.  110, 112);  FM  3-90.2  (p.  9-48  to  9-51); 

FM  44-100  (Chs.  4  and  6) 
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35.  Determine  direct  and  indirect  fire  systems  in  an  air  defense  role. 

Task  Definition:  Combined  arms  elements  can  provide  vital  self-protection  from  air  threats  and 
contribute  to  freedom  of  maneuver.  Although  they  have  a  limited  capability  to  engage  fixed- 
wing  aircraft,  missiles,  and  unmanned  aerial  vehicles  (UAVs),  combined  arms  elements  can 
effectively  engage  hovering  or  slow-moving  helicopters  within  their  weapon  systems'  ranges. 
Tank  main  guns,  infantry  fighting  vehicles  (IFV),  antitank  weapons,  and  other  direct-fire  systems 
must  engage  these  threat  air  platforms  when  possible.  Fire  support  enhances  tactical-level  air 
and  missile  defense.  Indirect  fire  weapons  can  deny  enemy  helicopters  the  use  of  masked, 
standoff  positions.  Fire  support  systems  can  concentrate  their  fires  on  enemy  landing  zones, 
pickup  zones,  launch  sites,  command  and  control,  assembly  areas,  and  forward  arming  and 
refueling  point  (FARP).  Considerations  for  determining  direct  and  indirect  fires  systems  in  an 
air  defense  role  include  (a)  the  nature  of  the  target;  (b)  the  distance  of  the  target;  (c)  target 
visibility;  and  (d)  the  desired  effect  of  fires. 

Staff  Officers  Involved:  S2,  S3,  FSO,  ADO 

Rationale:  The  task  force  commander  and  his  ADO  integrate  the  firepower  of  all  available  fire 
systems  to  defeat  the  enemy  air  threat.  The  ADO  works  with  the  S2  to  determine  air  threat.  The 
ADO  assists  the  S3  in  planning  and  executing  the  air  defense  portion  of  the  operation.  He 
coordinates  with  the  S3  Air  (if  assigned),  ALO  (at  the  brigade  level  and  above),  FSO,  and 
forward  air  controller  for  the  appropriate  air  defense  posture  and  Army  airspace  C2. 

References:  Commander’s  Battle  Staff  Handbook  (p.  60);  FM  3-90.2  (pp.  9-49  to  9-50);  FM  44- 
1 00  (Ch.  6);  FM  1 0 1  -5  (pp.  4-22,  4-23) 

36.  Define  air  defense  fratricide  prevention  criteria. 

Task  Definition:  The  lack  of  coordination  between  friendly  forces  is  one  of  the  major  factors 
causing  fratricide.  Air  defense  must  be  continually  synchronized  with  aviation  operations  to 
preclude  fratricide  of  friendly  aviation  assets.  Risk  of  fratricide  is  determined  first  by  identifying 
hazards,  then  by  assessing  each  hazard  to  determine  the  risk  of  potential  loss  based  on  the 
hazard’s  probability  and  the  severity.  Developing  controls  that  will  eliminate  or  reduce  the  risk 
of  the  hazard  requires  consideration  of  the  reason  for  the  hazard  and  specifying  the  who,  what, 
when,  where,  and  how  for  each  control. 

Staff  Officers  Involved:  XO,  S3,  ADO 

Rationale:  The  ADO  plans  and  coordinates  airspace  with  the  aviation  liaison  officer,  the  ALO 
(at  the  brigade  level  and  above),  the  S3  Air  officer  (if  assigned),  and  other  airspace  users.  The 
S3  aids  in  deconfliction  by  providing  input  on  the  effects  of  operations  for  airspace.  The  XO  is 
responsible  for  integrating  fratricide  countermeasures  into  the  plan. 

References:  Commander’s  Battle  Staff  Handbook  (pp.  55  to  61,  110);  FM  101-5  (p.  4-2,  4-22,  4- 
23,  Appendix  J) 
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BOS  #7  -  Combat  Service  Support  (CSS) 

37.  Determine  the  adequacy  of  the  area  for  CSS  operations. 

Task  Definition:  The  six  essential  CSS  functions  are  arm,  fuel,  fix,  move,  sustain,  and  man. 
Determination  of  the  adequacy  of  the  area  for  CSS  operations  therefore  requires  evaluation  of  the 
area’s  capability  to  support  the  safe,  secure,  and  rapid  transportation  and  delivery  of  supplies  (CL 
I,  II,  III,  III  (B),  IV,  V,  VII,  VIII,  and  IX  [see  Appendix  A]),  tools,  equipment,  and  personnel 
between  the  battalion  support  area,  the  task  force  support  area,  and  task  force  units. 

Staff  Officers  Involved:  XO,  SI,  S2,  S3,  S4,  ENG 

Rationale:  The  XO,  SI,  and  S4  are  the  principal  CSS  planners.  Assurance  of  safe,  secure,  and 
rapid  transportation  and  delivery  of  support  requires  that  CSS  planners  know  (a)  the  mission, 
task  organization,  and  concept  of  operations  for  all  subordinate  units  in  the  task  force;  (b)  known 
and  anticipated  branch  plans  and  sequels;  (c)  known  and  anticipated  enemy  situation  and 
capabilities.  The  S3  and  S2  assist  in  determining  the  adequacy  of  the  area  for  CSS  operations  by 
providing  the  principal  CSS  planners  this  information.  In  addition,  the  S2  and/or  ENG  provide 
useful  information  regarding  terrain  and  weather  implications  for  CSS  operations.  The  HHC  (or 
FSC)  CDR  and  the  surgeon  may  also  help  with  this  task. 

References:  Commander’s  Battle  Staff  Handbook  (pp.  56,  58,  74);  FM  3-90.2  (pp.  10-13  to  10- 
34) 

38.  Determine  transportation  requirements  and  priorities. 

Task  Definition:  The  determination  of  transportation  requirements  and  priorities  requires 
consideration  of  (a)  movement  of  CL  IV  and  V  obstacle  materials  and  engineer  equipment  to 
designated  work  sites  or  supply  points  by  the  required  times;  (b)  the  potential  to  evacuate  enemy 
materiel  and  personnel;  (c)  the  transportation  necessary  to  support  offensive  operations  with 
aerial  re-supply,  forward  positioning  of  CL  III  and  CL  V,  repositioning  of  other  supplies 
forward,  and  refuel-on-the-move  (ROM)  operations;  (d)  the  transportation  necessary  to  support 
defensive  operations  with  CL  IV  and  CL  V  items  and  engineer  equipment  for  defensive 
preparation,  repositioning  of  CL  IV  and  CL  V  items  and  engineer  equipment  to  subsequent 
defensive  positions,  evacuation  of  supplies  and  equipment  to  planned  fallback  points,  and 
evacuation  of  medical  units  with  alternate  means  of  transportation;  (e)  the  transportation 
necessary  to  support  anticipated  surge  requirements;  (f)  the  need  to  position  logistics  facilities; 

(g)  the  impact  of  extended  operations  or  line  of  communications  (LOCs)  on  driver/operator  rest 
factors;  (h)  the  impact  of  terrain  or  extended  operations  on  operators  and  readiness  of 
transportation  assets;  (i)  the  need  for  route  improvement;  (j)  movement  distances,  routes,  and 
required  delivery  times  to  work  sites/supply  points;  (k)  the  type  and  quantities  of  materials 
required  to  be  moved;  and  (1)  the  availability  of  special  equipment  (e.g.,  heavy  equipment  and 
transport). 


Staff  Officers  Involved:  XO,  S3,  S4,  ENG 


Rationale:  The  XO  and  S4  are  the  principal  CSS  planners  for  materiel.  Determination  of 
transportation  requirements  and  priorities,  as  described  above,  requires  that  CSS  planners  know 
(a)  the  mission,  task  organization,  and  concept  of  operations  for  all  subordinate  units  in  the  task 
force;  (b)  known  and  anticipated  branch  plans  and  sequels;  (c)  the  engineer  plan  and  material 
requirements  for  building  obstacles;  (d)  terrain  and  weather  implications  for  transportation;  and 
(e)  the  integrated  maneuver/fire  support  plan.  The  S3  and  ENG  assist  in  determining 
transportation  requirements  and  priorities  by  providing  the  principal  materiel  planners  this 
information.  The  battalion/task  force  maintenance  officer  is  a  critical  asset  in  the  S4  section 
supporting  this  task. 

References:  ARTEP  63-216-MTP;  ARTEP  71-3-MTP;  CGSC  ST  101-6  (pp.  1-11); 
Commander’s  Battle  Staff  Handbook  (pp.  66  to  72);  FM  3-90.2  (pp.  10-26  to  10-34);  FM  90-7 
(Appendix  C) 

39.  Determine  medical  support  requirements. 

Task  Definition:  Determining  medical  support  requirements  involves  estimating  casualties, 
including  task  force  scouts  and  other  forward  reconnaissance  elements,  and  defining  medical 
evacuation  (MEDEVAC)  support. 

Staff  Officers  Involved:  XO,  S1,S4 

Rationale:  The  XO,  SI,  and  S4  are  the  principal  CSS  planners,  with  the  SI  having  medical 
planning  and  casualty  management  as  key  staff  responsibilities.  The  surgeon  is  an  asset  in  the 
SI  section  who  helps  to  determine  medical  support  requirements. 

References:  Commander’s  Battle  Staff  Handbook  (pp.  28,  31-34);  FM  3-90.2  (p.  10-26, 10-30) 

40.  Identify  points  in  the  battle  when  surge  requirements  are  likely  to  be  generated. 

Task  Definition:  Surge  requirements  are  likely  to  be  generated  at  such  points  in  the  battle  as  (a) 
seizing  an  objective  at  the  completion  of  an  attack  (e.g.,  surge  to  reposition  critical  supplies  for 
consolidation  and  reorganization  or  surge  in  CL  III  and  CL  V  to  support  unexpected 
success/pursuit  of  the  enemy);  (b)  initiation  of  hasty  defense  or  mission  change  from  offense  to 
defense  (e.g.,  surge  in  CL  IV,  CL  V,  and  engineer  equipment  to  support  defense  preparation); 
and  (c)  attack  of  prepared  defensive  position  with  major  obstacles  (e.g.,  potential  surge  in 
medical  personnel,  supplies,  and  facilities  to  support  a  mass  casualty  situation).  Identifying  these 
or  other  points  of  likely  surge  requirements  involves  anticipating  the  implications  of  friendly  and 
enemy  actions  (expected  and  unexpected)  for  personnel  and  supply  requirements.  The 
battalion/task  force  maintenance  officer  is  a  critical  asset  in  the  S4  section  supporting  this  task. 

Staff  Officers  Involved:  XO,  S2,  S4 

Rationale:  The  XO  and  S4  are  the  principal  CSS  planners  for  materiel.  The  S4  section  is 
responsible  for,  among  other  things,  providing  ammunition,  fuel,  food,  water,  maintenance,  and 
transportation  services  to  companies,  conducting  emergency  re-supply,  and  anticipating  supply 
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requirements.  Interaction  with  the  S2  provides  the  intelligence  required  to  forecast  losses  and 
subsequent  re-supply.  At  higher  echelons  (DIV,  CORPS),  the  SI  is  involved  in  planning  for 
surge  requirements  for  personnel.  The  battalion/task  force  maintenance  officer  is  a  critical  asset 
in  the  S4  section  supporting  this  task. 

References:  Commander’s  Battle  Staff  Handbook  (pp.  65  to  77);  FM  3-9.2  (p.  10-26) 

41.  Determine  tactical  restrictions  on  Combat  Service  Support  operations. 

Task  Definition:  Tactical  restrictions  include  (a)  terrain  that  cannot  be  used  for  logistics 
operations  because  it  is  being  occupied  by  a  tactical  unit;  (b)  roads  which  are  dedicated  to  a 
company’s  movement  to  the  LD  during  a  specific  period  and  not  available  for  use  by  logistical 
vehicles;  and  (c)  weather  conditions  that  prevent  logistics  operations  in  specific  areas  due  to 
trafficability. 

Staff  Officers  Involved:  XO,  S3,  S4,  ENG 

Rationale:  The  XO  and  S4  are  the  principal  CSS  materiel  planners.  The  S4  section  is 
responsible  for,  among  other  things,  providing  ammunition,  fuel,  food,  water,  maintenance,  and 
transportation  services  to  companies,  conducting  emergency  re-supply,  and  anticipating  supply 
requirements.  Determination  of  the  tactical  restrictions  on  CSS  operations  involves  integration 
of  the  CSS  plan  with  the  maneuver  plan,  a  shared  responsibility  of  the  CSS  planners  and  the  S3. 
The  ENG  can  provide  useful  information  regarding  the  effects  of  weather  conditions  on  the 
trafficability  of  support  vehicles.  The  battalion/task  force  maintenance  officer  is  a  critical  asset 
in  the  S4  section  supporting  this  task. 

References:  Commander’s  Battle  Staff  Handbook  (pp.  56,  65  to  77);  FM  3-9.2  (p.  10-26);  LL- 
CALL  Newsletter  No.  88-3  (p.  31) 

42.  Compare  required  and  available  Combat  Service  Support  capability  to  identify 
shortfalls  and  ways  and  means  to  mitigate  the  effect  of  these  shortfalls. 

Task  Definition:  Comparison  of  required  and  available  CSS  capability  requires  integration  of 
the  CSS  plan  with  the  integrated  maneuver  plan. 

Staff  Officers  Involved:  XO,  S2,  S3,  S4,  SIGO,  FSO,  ADO,  ENG 

Rationale:  The  XO,  S3,  and  S4  coordinate  closely  to  ensure  that  tactical  plans  are  logistically 
supportable.  The  S4  section  is  responsible  for,  among  other  things,  providing  ammunition,  fuel, 
food,  water,  maintenance,  and  transportation  services  to  companies,  conducting  emergency  re¬ 
supply,  and  anticipating  supply  requirements.  Every  staff  member  is  responsible  for  identifying 
requirements  for  additional  units,  equipment,  or  support  in  their  areas  of  interest,  though  the  SI 
becomes  more  involved  in  planning  personnel  requirements  at  higher  echelons  (DIV,  CORPS). 
The  ENG  and  FSO  provide  information  as  to  what  support  and  transportation  priority  they  may 
need  during  the  operation.  The  battalion/task  force  maintenance  officer  is  a  critical  asset  in  the 
S4  section  supporting  this  task. 
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References:  Commander’s  Battle  Staff  Handbook  (pp.  56,  58,  65  to  77);  FM  3-9.2  (p.  10-26) 
FM  101-5  (p.  4-6) 
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Appendix  C 

Wargaming  Tacit  Knowledge  Assessment 
Note:  Correct  answers  have  been  bolded. 

1 .  According  to  FM  5-0,  wargaming  is,  in  part,  a  tool  to  help  staffs  determine  the  strengths  and 
weaknesses  of  multiple  courses  of  action  (CO As).  Then,  the  multiple  COAs  are  compared  and  a 
“best”  COA  selected.  Time  is  often  limited,  however,  and  staffs  are  given  only  a  single  COA  to 
war  game.  What  is  the  purpose  of  wargaming  a  single  COA? 

a.  To  update  staff  estimates,  based  on  what  is  learned  in  the  wargame. 

b.  To  rehearse  the  mission,  testing  assumptions  about  the  enemy  and  terrain. 

c.  To  refine  the  plan  and  synchronize  the  battlefield  operating  systems. 

d.  To  ensure  that  the  commander’s  COA  is  the  best  way  to  accomplish  the  mission. 


2.  The  wargaming  process  is  conducted  using  multiple  iterations  of  an  action-reaction- 
counteraction  (ARC)  cycle.  Why? 

a.  The  iterative  ARC  cycle  involves  a  logical  sequence  of  action. 

b.  The  iterative  ARC  cycle  reveals  the  impact  of  timing  on  friendly  and  enemy  action. 

c.  The  iterative  ARC  cycle  supports  wargaming  both  offensive  and  defensive  battles. 

d.  The  iterative  ARC  cycle  simulates  how  the  mission  will  play  out  if  the  enemy  follows  a 
particular  (e.g.,  most  probable)  COA. 


3.  What  is  the  purpose  of  estimating  losses  during  the  wargaming  process? 

a.  To  calculate  relative  combat  power  during  each  phase  of  the  operation. 

b.  To  determine  materiel  shortages  (e.g.,  breaching  assets). 

c.  To  establish  medical  supply  and  transportation  requirements. 

d.  To  visualize  how  losses  will  affect  the  commander’s  decision-making. 


4.  Each  of  the  seven  BOSs  are  considered  during  the  wargaming  process.  Why? 

a.  So  every  staff  officer  is  aware  of  each  other’s  responsibilities. 

b.  So  the  integrated  contribution  of  each  BOS  to  the  fight  can  be  determined. 

c.  So  each  staff  officer  can  determine  what  he  has  to  track  during  the  battle. 

d.  So  the  BOSs  can  be  deconflicted. 
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5.  What  is  a  key  difference  between  a  synchronization  matrix  and  a  COA  statement/sketch? 

a.  A  synch  matrix  has  more  detail. 

b.  A  synch  matrix  includes  all  of  the  BOSs. 

c.  A  synch  matrix  reflects  a  refined  COA. 

d.  A  synch  matrix  better  illustrates  how  mission  events  will  happen  over  time. 
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Appendix  D 

Team-Related  Motivation  Survey 


Item 

Staff  positions  provide  experiences  that  are  important  for  developing 
command  skills. 

Utility  of  Performance 

Good  performance  in  a  staff  position  is  required  for  promotion  to 
command. 

Utility  of  Performance 

Great  staff  members  don’t  make  history;  Great  commanders  do. 

Utility  of  Performance 

Superior  staff  performance  is  a  source  of  pride  for  individual  staff 
members. 

Utility  of  Performance 

Good  commanders  don’t  need  a  staff  to  aid  in  planning;  They  can 
already  visualize  the  battlefield  and  act  decisively. 

Utility  of  Performance 

Staff  performance  is  not  a  major  factor  in  determining  battle 
outcomes. 

Utility  of  Performance 

Technical  knowledge  (including  doctrine  and  digital  skills)  is  critical 
for  superior  staff  performance. 

Utility  of  Performance 

Improved  technical  knowledge  is  worth  the  effort  involved  in 
acquiring  it. 

Perceived  Effort- 
Performance  Relation 

Team  cohesion  is  a  critical  characteristic  of  superior  staffs. 

Utility  of  Performance 

Improved  team  cohesion  is  worth  the  effort  involved  in  developing  it. 

Perceived  Effort- 
Performance  Relation 

There  is  higher  payoff  for  investing  effort  to  acquiring  technical 
knowledge  than  for  investing  effort  in  developing  team  cohesion. 

Perceived  Effort- 
Performance  Relation 

Staff  performance  can  only  be  so  good;  External  factors  (e.g.,  rapid 
staff  turnover)  exert  a  strong  limiting  influence. 

Utility  of  Effort 

^Motivation  aspect  was  not  shown  in  the  survey  administered  to  researc 

i  participants,  but  is 

shown  here  for  reader  reference.  Rating  scale  was  Definitely  True,  Largely  True,  Depends, 
Largely  False,  Definitely  False. 
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Appendix  E 

Team  Communication  Checklist 
Distributed  War- Gaming,  Task  Force  1-93  Scenario 
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members  of  the  local  populace,  such  as 
holy  days;  times  when  friendly  forces  are 
distracted  by  other  engagements  such  that 
they  can't  sufficiently  address  the  problem). 
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Appendix  G 


Situation  Awareness  Exercise 


SA  Level 

Question 

Answer 

1 

What  is  the  center  of  mass  for  NAB? 

719946 

2 

If  the  enemy  strongpoint  is  located  forward  on 
high  ground  at  NAB,  what  would  this  reveal 
about  the  enemy  intends  to  do? 

Protect  that  key  piece  of  terrain 
or  die  trying 

3 

What  should  TF  1-93  do  if  the  eastern  enemy 
strongpoint  is  located  forward  on  high  ground? 

Establish  two  teams  in  SBF 
position  to  engage  the 
strongpoint  while  directing  fires 
into  the  western  mechanized 
infantry  company  (MIC) 

1 

What  is  the  center  of  mass  for  NAM? 

702956 

2 

If  there  are  any  dismounted  infantry  forces 
forward  in  sector,  spotted  at  NAM,  what  would 
this  reveal  about  what  the  enemy  intends  to  do? 

The  enemy  intends  to  fight  the 
combat  security  outpost  (CSOP) 
forward  in  sector,  and  to  use  the 
mechanized  forces  of  the  eastern 
MIC  as  reinforcements 

3 

What  should  TF  1-93  decide  to  do  if  this 
happens,  and  where  should  this  decision  be 
made  (i.e.,  DP  center  of  mass)? 

Fix  eastern  MIC  with  lead  TM 
and  maneuver  remaining  teams 
toward  the  western  end  of  the 
objective  (DP  center  of  mass  = 
696893) 

1 

At  what  location  (i.e.,  center  of  mass)  should  the 
disposition  of  the  AT  battery  be  observed? 

New  NAI,  center  of  mass  = 

694974 

2 

If  the  enemy  begins  displacing  an  AT  platoon  at 
this  location,  what  would  this  reveal  about  what 
he  intends  to  do? 

Commit  a  counterattack;  expect 
considerable  flanking  force 
activity  from  the  North  West 
(NW) 

3 

What  should  TF  1-93  decide  to  do  if  this 
happens,  and  where  should  this  decision  be 
made  (i.e.,  DP  center  of  mass)? 

Fix  AT  platoon  with  indirect 
fires;  destroy  with  CAS  (DP 
center  of  mass  =  694961) 

1 

At  what  location  (i.e.,  center  of  mass)  should  the 
commitment  of  the  eastern  MIC  be  watched  for? 

New  NAI,  center  of  mass  = 

725923 

2 

If  the  enemy  holds  mechanized  forces  from  the 
eastern  MIC  in  reserve  and  maneuvers  around 
the  eastern  flank,  what  would  this  reveal  about 
his  intentions? 

He  is  likely  trying  to  assail  a 
flank  and  run  you  against  another 
stationary  force  to  your  west 

3 

What  should  TF  1-93  decide  to  do  if  this 
happens,  and  where  should  this  decision  be 
made  (i.e.,  DP  center  of  mass)? 

Use  one  company  team  in  SBF 
to  respond  (branch  plan  A)  (DP 
center  of  mass  =  716935) 

G-l 


1 

At  what  location  (i.e.,  center  of  mass)  or 
locations  should  the  commitment  of  the  enemy 
reserve  be  watched  for? 

New  NAI,  center  of  mass  = 

666943  or  726978  or  previous 
new  NAI,  center  of  mass  = 

694974 

2 

If  the  enemy  commits  a  reserve  to  the  TF  1  -93 
sector,  what  would  this  reveal  about  his 
intentions? 

He  cannot  fail  in  this  sector  (this 
is  the  last  line  of  his  organic 
forces/defense  in  depth) 

3 

What  should  TF  1  -93  do  if  this  happens,  and 
where  should  this  decision  be  made  (i.e.,  DP 
center  of  mass)? 

Use  assault  force  on  PITBULL 
to  fix  the  reserve;  maneuver  SBF 
teams  to  destroy  it  (DP  center  of 
mass  =  710981) 

G-2 


Appendix  H 

Integrated  Overlay  Exercise  Items  and  Rationale 
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Overlay  Element _ Rationale/Justification _  Staff  Officer  Responsible 
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(.5  point  each) 


Score  -  TOTAL  actual  number  of  points  earned 
TOTAL  maximum  number  of  points 


Appendix  I 


Demographic  Survey 


General  Information  (please  fill  out  the  table  below  by  typing  your  response  in  the  right-hand  column) 


Question  Answer 


1.  Age: 


2.  Staff  Position  in  AC3DL  (e.g.,  XO 


3.  Basic  Branch  (e.g.,  AR,  IN,  FA,  etc.): 


4.  List  Other  Basic  Branch  Experience: 
(indicate  months  of  experience  in  each  branch) 


5.  Current  Rank: 


6.  Current  Duty  Position: 


7.  Months  in  Current  Duty  Position: 


8.  Years  of  Regular  Army  Experience: 


9.  Highest  Enlisted  Rank  Achieved: 

(type  “N/A”  if  question  is  not  applicable  to  you) 


[  Command  (please  fill  out  the  table  below,  where  applicable;  select  Yes  or  No  by  italicizing  it): 


!  CTC  Rotation? 

COMPANY  NAME  Months  as  Commander 


Yes 

No 

Yes 

No 

Yes 

No 

Prior  Staff  Experience  (please  fill  out  table  below,  where  applicable;  select  Yes  or  No  by  italicizing  it): 


Staff  Position  (e.g.,  Company  XO) 

Months  in  Position 

CTC  Rotation? 

11a. 

Yes 

No 

Yes 

No 

lib. 

Yes 

No 

Yes 

No 

11c. 

Yes 

No 

Yes 

No 

lid. 

Yes 

No 

Yes 

No 

GO  ON  TO  THE  NEXT  PAGE 
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If  you  have  NOT  had  prior  staff  experience,  please  skip  to  Question  17.  If  you  listed  prior 
staff  positions  in  the  table  above,  please  go  on  to  Question  12. 


12.  While  serving  in  the  above-listed  staff  positions,  did  you  participate  in  a  COA  Analysis  (i.e., 
war  game)?  (please  italicize  your  answer) 

Yes  No 

13.  Where  have  you  conducted  COA  Analysis?  [you  may  choose  more  than  one  -  please  italicize  your 
selection(s)] 

Home-Station  TX  CTC  Rotation  Deployment 

14.  Have  you  used  computer  simulation  to  conduct  COA  Analysis?  (please  italicize  your  selection) 

Yes  No 

15.  Have  you  used  digital  displays  during  COA  Analysis?  (please  italicize  your  selection) 

Yes  No 

16.  Have  you  participated  in  COA  Analysis  as  part  of  an  Abbreviated  MDMP  Process? 

Yes  No  (please  italicize  your  selection) 

1 7.  Other  Military  Courses  Taken  (please  italicize  your  selections): 

Air  Borne/ Air  Assault 
Ranger/Special  Forces 
BMOC/Other  CSS 
SPLC 

TC3/NBC  Defense/Other  (please  list  “Other”  Courses  below) 

18.  Have  you  had  experience  working  with  anyone  else  in  the  AC3DL  course  prior  to  taking  the 
course? 

Yes  No  (please  italicize  your  selection) 


GO  ON  TO  THE  NEXT  PAGE 
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Appendix  J 


General  Design  Guidelines  for  Scenario-Specific  Assessments 

The  wargaming  process  and  outcome  assessments  developed  in  the  present  research  are 
widely  applicable,  but  their  particular  instantiation  is  scenario-specific,  and  must  be  so  if  the  goal 
is  to  assess  situated  task  performance  (as  opposed  to  abstract,  general  constructs,  such  as  general 
intelligence  or  working  memory).  For  this  reason,  general  guidelines  for  designing  wargaming 
process  and  outcome  assessments  for  specific  scenarios  are  outlined  below.  Assessments 
analogous  to  those  used  in  the  present  research  can,  with  some  effort,  be  developed  to  assess  the 
same  elements  of  the  wargaming  framework  (e.g.,  battlefield  visualization)  using  different 
scenarios. 


General  Guidelines 


The  development  of  scenario-specific  assessments  of  wargaming  effectiveness  requires 
the  joint  participation  of  an  assessment/measurement  subject  matter  expert  and  one  or  more 
Army  subject  matter  experts.  The  assessment/measurement  expert  must  be  capable  of 
conceptualizing  assessment  tools  that  do  not  fit  well  with  traditional  psychometric  models  and 
able  to  communicate  arcane  assessment  criteria  to  a  non-expert.  The  Army  expert(s)  must  have 
relatively  recent  Army  experience  and  be  current  with  new  doctrine  and  relevant  organizational 
changes  in  the  Army.  He  (she)  must  also  be  capable  of  articulating  the  observable  behaviors 
associated  with  effective/ineffective  performance.  A  current  Army  professional  is  not  ideal  for 
assessment  development  due  to  the  time  demands  involved,  but  recently  retired  Army  officers 
are  a  great  resource  and  often  willing  to  help.  Contractors  working  as  Army  instructors  are  also 
a  very  helpful  resource  as  they  have  extensive  experience  with  observing,  thinking  about,  and 
evaluating  performance. 


Developing  an  Integrated  Plan/Overlay  Exercise 

Assessing  the  degree  of  integration  and  synchronization  in  the  mission  plan  refined  by 
wargaming  requires  determining  (a)  the  graphics  must  be  present  in  the  plan  overlay  in  order  to 
demonstrate  that  integration  across  BOSs  has  been  accomplished  and  (b)  the  rationale  criteria 
that  should  be  used  to  position  these  graphics. 

Determining  the  graphics  that  must  be  present  in  the  integrated  overlay  involves, 
essentially,  mentally  (or  literally,  if  there  is  access  to  enough  experts)  wargaming  the  scenario  for 
all  aspects  of  the  mission,  including  combat  support  (CS)  and  combat  service  support  (CSS) 
battlefield  operating  systems  (BOS).  For  assessing  the  staff  integration  of  combined  arms  units, 
special  attention  must  be  paid  to  CS  and  CSS,  such  as  obstacles,  supply  routes,  decontamination 
points,  etc.,  which  tend  to  get  left  out  of  wargaming  and  are  developed  in  a  stove-piped  manner 
to  produce  the  task  force  operations  order.  Additional  important  graphics  to  consider  include 
command  posts,  key  terrain,  and  target  reference  points.  Non-maneuver  officers  can  be 
especially  helpful  in  determining  these  critical  graphics  because  they  have  considerable 
experience  with  CS  or  CSS  aspects  of  operations 
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Determining  the  rationale  criteria  used  to  position  the  graphics  must  come  from 
consultation  with  doctrine  and  Army  experts.  To  a  noteworthy  degree,  these  criteria  can  be 
transferred  across  scenarios,  provided  the  critical  graphics  present  in  the  overlays  are  the  same. 

Materials  Required: 

1)  Detailed  brigade  operations  order 

2)  Commander's  planning  guidance  and  intent 

3)  Task  force  course-of-action  statement  and  sketch 

4)  Map  of  area 

5)  Doctrinal  materials  (e.g.,  FM  3-0,  Operations;  FM  3-90,  Tactics) 

Developing  a  Situation  Awareness  Exercise 

Assessing  battlefield  visualization/situation  awareness  using  a  modified  decision  support 
matrix  (DSM)  requires  determining  ahead  of  time  (again,  either  by  mental  or  actual  wargaming 
of  the  scenario)  the  number  and  map  position  of  the  critical  (a)  named  areas  of  interest  (NAI),  (b) 
target  areas  of  interest  (TAI),  (c)  and  decision  points  (DP)  that  should  be  included  in  the  matrix, 
as  well  as  (d)  the  indicators  of  enemy  decisions,  and  (e)  the  friendly  actions  that  must  be  taken  if 
particular  enemy  decisions  are  made.  (Included  below  is  a  blank  representation  of  an  example 
modified  DSM  that  can  be  used  as  a  template.)  Maneuver  officers  are  especially  helpful  making 
these  determinations,  as  they  have  considerable  experience  developing  maneuver  schemes. 

Note,  however,  that  a  sampling  of  multiple  experts,  especially  if  they  come  from  different 
branches,  may  produce  a  different  selection  of  elements  to  include  in  the  DSM.  Justifiable 
decisions  must  be  made  as  to  what  should  serve  as  a  “correct”  answer  in  each  cell  of  the  matrix. 

Materials  Required: 

1)  Situation  Template 

2)  Task  force  course-of-action  statement  and  sketch 

3)  Modified  Combined  Obstacle  Overlay 

4)  Map  of  area 

5)  Doctrinal  materials  (e.g.,  FM  3-0,  Operations-,  FM  3-90,  Tactics) 


Enemy 

Indicators 

— 

mmm 

DP 

(Center  of  Mass) 

Friendly  Action 

Developing  a  Think  Like  a  Commander  (TLAC)  Checklist 

Assessing  adaptivity  of  team  thought  by  evaluating  examinee  responses  to  TLAC  probe 
questions  requires  creating  the  probe  questions  and  determining  effective  and  ineffective 
responses  to  these  questions,  to  be  used  as  anchors  for  an  observer  rating  scale. 
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The  basis  of  the  probe  questions  is  the  eight  TLAC  themes  (listed  below).  The  kind  of  thinking 
that  questions  based  on  each  theme  required  is  also  listed  below.  The  listing  of  these 
requirements  provides  guidance  for  how  the  scenario-specific  probe  questions  should  be 
designed  via  a  combined  effort  of  Army  and  assessment/measurement  experts. 

Creating  probe  questions  at  an  adequate  difficulty  level  can  be  tricky.  They  should  be 
difficult  enough  to  challenge  the  examinee  to  think  beyond  what  can  be  regurgitated  from  the 
brigade  operations  order,  but  not  so  difficult  that  they  require  significant  time  to  generate  a 
response.  Examinees  should  be  able  to  demonstrate  flexibility  of  thought  without  getting  bogged 
down  in  the  weeds,  otherwise  the  process  will  take  too  long. 

Determining  effective  and  ineffective  responses  to  the  probe  questions  requires 
consultation  with  doctrine  and  subject  matter  experts.  A  combination  of  maneuver  and  non¬ 
maneuver  experts  is  the  best  arrangement,  if  possible. 

Materials  Required: 

1)  Detailed  brigade  operations  order 

2)  Task  force  course-of-action  statement  and  sketch 

3)  The  eight  TLAC  themes  (these  themes  and  the  nature  of  the  questions  associated  with 
each  are  listed  below) 

4)  Situation  Template 

5)  Map  of  area 

6)  Doctrinal  materials  (e.g.,  FM 3-0,  Operations;  FM  3-90,  Tactics) 


The  Eight  TLAC  Themes  and  the  Nature  of  their  Probe  Questions 
Keep  a  Focus  on  the  Mission  and  Higher ’s  Intent 

Probe  questions  under  this  theme  challenge  the  students  to  apply  their  understanding  of 
mission  and  intent  to  address  a  change  in  events 

Model  a  Thinking  Enemy 

Probe  questions  under  this  theme  challenge  the  students  to  (a)  apply  their  knowledge  of 
the  enemy  to  consider  branches  to  the  COA  or  alternative  COAs;  (b)  describe  how  they  would 
commit  forces  if  they  were  the  defending  enemy  (rather  than  attacking  force);  or  (c)  describe 
how  they  would  handle  enemy  deployment  of  NBC  weapons. 

Consider  Effects  of  Terrain 

Probe  questions  under  this  theme  challenge  the  students  to  (a)  apply  their  knowledge  of 
the  terrain  to  consider  alternative  actions  to  accomplish  the  mission  if  terrain  changes  (e.g.,  a 
new  obstacle  is  discovered,  concealment  is  destroyed  by  enemy  countermobility,  key  pass  is 
blocked,  civilian  demonstration  takes  place);  (b)  consider  rates  of  movement  for  various  types  of 
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terrain  under  differing  environmental  conditions;  and  (c)  apply  their  knowledge  of  terrain  to 
justify  maneuver  COAs  on  the  basis  of  terrain  available  to  them/to  the  defender. 

Use  All  Assets  Available 

Probe  questions  under  this  theme  challenge  the  students  to  (a)  apply  their  knowledge  of 
the  range  of  assets  available  to  address  a  change  in  their  expected  level  of  resources;  and  (b) 
demonstrate  general  knowledge  of  the  capabilities  or  functions/assets  from  other  BOS’s,  such  as 
CS  and  CSS,  so  that  these  assets  can  be  appropriately  positioned/effects  maximized. 

Consider  Timing 

Probe  questions  under  this  theme  challenge  the  students  to  apply  their  sense  of  timing  to 
synchronize  likely  battlefield  events  (e.g.,  overlapping  casualty  evacuation  (CASEVAC)  with 
fires,  how  long  to  take  care  of  CASEVAC  before  fires  can  be  issued,  passage  of  lines  for  a 
maneuver  battalion  through  limited  passage  lanes) 

See  the  Big  Picture 

Probe  questions  under  this  theme  challenge  the  students  to  (a)  apply  their  knowledge  of 
the  entire  Bde  mission  to  recognize  what  implications  the  success  (or  failure)  of  other  TF  units 
has  for  their  own  TF  mission  and  vice  versa;  and  (b)  respond  to  the  destruction  of  their  own 
mobility  reserve. 

Visualize  the  Battlefield 

Probe  questions  under  this  theme  challenge  the  students  to  (a)  describe  the  impacts  of 
terrain  (or  the  enemy)  on  friendly  mobility/maneuver;  (b)  describe  fields  of  fire  for  friendly  or 
enemy  weapons  systems;  (c)  identify  where  secondary  positions  or  alternate  locations  exist  for 
someone  defending  in  depth.  NOTE  -  This  list  is  not  in  any  way  exhaustive.  Probe  questions 
for  the  above  6  TLAC  themes,  and  the  below  theme  “Consider  Contingencies  and  Remain 
Flexible”  can  all  fall  under  the  rubric  of  “Visualize  the  Battlefield.” 

Consider  Contingencies  and  Remain  Flexible 

Probe  questions  under  this  theme  challenge  the  students  to  (a)  respond  effectively  to 
unexpected  events;  (b)  consider  NAI/TAIs  to  flexibly  assess  and  respond  to  where  the  enemy 
might  go/what  the  enemy  might  do;  and  (c)  consider  seams/flanks  in  the  formation  as  possible 
vulnerabilities. 

Developing  a  Team  Communication  Checklist 

Assessing  team  communication  by  evaluating  staff  information  sharing  during 
wargaming  requires  determining  (a)  which  wargaming  tasks  to  target  for  observation;  (b)  which 
staff  officers  should  be  sharing  information  during  those  tasks;  and  (c)  what  information  must  be 
shared  in  order  for  the  task  to  be  completed  effectively. 
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Determining  the  team  communication  target  tasks  can  be  tricky.  Although  a 
comprehensive  list  of  tasks  to  be  completed  during  brigade-level  wargaming  has  been  developed 
(Mullen  et  al.,  1997),  staffs  often  do  not  complete  many  of  these  tasks  during  wargaming  (if,  in 
operational  environments,  they  war  game  at  all).  That  said,  this  list,  modified  for  task-force- 
level  planning,  can  serve  as  the  basis  for  selecting  the  team  communication  target  tasks.  A 
subset  of  this  list  of  42  wargaming  tasks  can  make  useful  points  of  observation  (routine-event 
targets)  for  staff  information  sharing.  Wargaming  tasks  of  special  interest  are  those  identified  as 
needing  emphasis  in  the  Center  of  Army  Lessons  Learned  analyses  of  trends  in  Combat  Training 
Center  command  and  control  team  performance  (e.g.,  integrate  fire  support  with  maneuver  and 
priorities).  In  addition,  tasks  that  have  some  likelihood  of  being  completed  during  the  war  game 
should  also  be  selected.  For  example,  some  of  the  42  wargaming  tasks  are  best  used  for 
defensive  operations  (e.g.,  determine  ways  and  means  to  separate  attacking  enemy  echelons)  and 
so  should  not  be  used  when  the  staff  is  wargaming  an  offensive  operation.  Determining  the 
likelihood  that  a  task  will  be  attempted  during  wargaming  can  be  aided  by  comparing  the 
synchronization  matrices  of  previous  wargame  s  using  the  same  scenario,  if  available,  to  the 
brigade  operations  order  (additional  elements  in  the  synch  matrix  were  tasks  completed  during 
wargaming),  or  by  observing  the  tasks  accomplished  in  other  war  games,  or  consulting  with 
subject  matter  experts.  Note  that  doctrine  is  not  helpful  in  determining  the  likelihood  that  a  task 
will  be  attempted.  This  is  because  doctrine  represents  what  should  ideally  be  done,  rather  than 
what  is  actually  done. 

Determining  the  staff  officers  who  should  share  information  for  each  wargaming  task  is 
made  easy  by  the  research  conducted  in  the  present  research.  The  necessary  staff  officers  for 
each  task  are  listed  below,  along  with  the  42  task-force-level  wargaming  tasks  themselves.  Note 
that  this  list  focuses  on  core  staff  officers,  as  is  therefore  not  exhaustive.  It  also  may  require 
some  modification  for  different  kinds  of  units.  This  list  is,  however,  doctrine  based  and 
verifiable. 

Determining  the  information  shared  for  each  task  involves  exploration  of  doctrine  and 
consultation  with  subject  matter  experts.  A  combination  of  maneuver  and  non-maneuver  experts 
is  the  best  arrangement,  if  possible.  The  information  shared  is  somewhat  portable  across 
scenarios,  provided  the  wargaming  tasks  identified  for  each  scenario  are  the  same. 

Materials  Required: 

1)  Detailed  brigade  operations  order 

2)  Task  force  course-of-action  statement  and  sketch 

3)  List  of  the  42  wargaming  tasks  to  be  accomplished  at  the  task  force  level  (included 
below) 

4)  Map  of  area 

5)  Modified  Table  of  Organization  &  Equipment  or  a  SMARTBOOK  reference  of  critical 
BOS  systems  and  their  capabilities 

6)  Recent  Center  of  Army  Lessons  Learned  trends  analyses 
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The  42  Wargaming  Tasks  to  be  Accomplished  at  the  Task  Force  Level  (see  Mullen  et  al.,  1997) 

1 .  Clearly  identify  the  commander’s  intent  and  vision  of  the  battle. 

XO,  SI,  S2,  S3,  S4,  SIGO,  FSO,  ADO,  ENG 

2.  Determine  the  command  and  control  requirements  for  minimizing  the  effect  of  destruction  of 
friendly  CPs. 

XO,  S3,  SIGO 

3.  Determine  or  refine  the  CCIR. 

XO,  SI,  S2,  S3,  S4,  SIGO,  FSO,  ADO,  ENG 

4.  Determine  critical  events  and  decision  points. 

XO,  SI,  S2,  S3,  S4,  SIGO,  FSO,  ADO,  ENG 

5.  Determine  command  post  locations  and  composition  to  support  current  and  planned  tactical 
operations. 

XO,  SI,  S2,  S3,  S4,  SIGO,  ENG 

6.  Assess  the  potential  effect  of  battle  intensity  on  Soldier  and  leader  will  to  fight. 

XO,Sl 

7.  Determine  HPTs. 

XO,  S2,  S3,  S4,  FSO,  ADO,  ENG 

8.  Determine  optimal  times  and  locations  to  maximize  enemy  casualties  and  force  destruction. 
XO,  S2,  S3,  S4,  FSO,  ADO,  ENG 

9.  Determine  ways  and  means  to  separate  attacking  enemy  echelons. 

XO,  S2,  S3,  S4,  FSO,  ADO,  ENG 

10.  Determine  ways  and  means  to  force  enemy  elements  into  areas  where  the  commander  wants 
them. 

XO',  S2,  S3,  S4,  FSO,  ADO,  ENG 

1 1 .  Identify  expected  enemy  air  or  helicopter  threats. 

S2,  ADO 

12.  Determine  optimal  employment  of  intelligence  collection  assets. 

XO,  S2,  S3,  S4,  SIGO,  FSO,  ADO,  ENG 

13.  Define  branches  and  sequels  to  the  maneuver  scheme. 

XO,  S2,  S3,  S4,  SIGO,  FSO,  ADO,  ENG 

14.  Identify  triggers  for  the  initiation  of  direct  and  indirect  fires. 

S2,  S3,  S4,  FSO,  SIGO,  ENG 

15.  Determine  route  prioritization  for  movement. 

52,  S3,  S4,  ADO,  ENG 

16.  Define  task  organization  requirements. 

XO,  SI,  S2,  S3,  S4,  SIGO,  FSO,  ADO,  ENG 

1 7.  Define  force  protection  criteria. 

XO,  SI,  S2,  S3,  S4,  SIGO,  FSO,  ADO,  ENG 

18.  Determine  effect  of  limited  visibility  on  combat,  CS,  and  CSS  operations. 

XO,  S2,  S3,  S4,  FSO,  ADO,  ENG 

19.  Integrate  fire  support  with  maneuver  and  priorities. 

53,  S4,  FSO 

20.  Update  HPTL. 

XO,  S2,  S3,  FSO,  ADO,  ENG 
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21.  Synchronize  lethal  and  nonlethal  fires  to  support  task  force  ISR  operations. 

52,  S3,  S4,  FSO 

22.  Synchronize  lethal  and  nonlethal  fires  to  support  task  force  offensive  operations. 

53,  S4,  FSO 

23.  Develop  an  observation  plan  that  assigns  (a)  responsibilities  to  target  acquisition  systems; 
and  (b)  observers  for  the  employment  of  indirect  fires  against  designated  targets  and 
determination  of  damage  assessments. 

S2,  S3,  SIGO,  FSO 

24.  Verify  sensor  taskings  to  provide  targetable  intelligence  in  a  timely  manner  for  HPTs. 

S2,  S3,  FSO 

25.  Assess  potential  enemy  actions  against  the  task  force’s  efforts  to  bypass  or  overcome 
obstacles. 

S2,  S3,  FSO,  ENG 

26.  Define  reconnaissance  requirements  to  identify  points  of  penetration  into  enemy  obstacles 
and  river  crossing  sites. 

52,  S3,  S4,  FSO,  ENG 

27.  Integrate  engineers  into  maneuver  formations  to  maintain  momentum,  with  the  bulk  of 
mobility  assets  with  the  breach  force. 

53,  S4,  FSO,  ENG 

28.  Determine  FASCAM  employment. 

S2,  S3,  FSO,  ENG 

29.  Define  emplacement  criteria  for  obstacles  and  mines  to  protect  the  task  force  flanks  and 
block  enemy  counterattacks. 

S2,  S3,  S4,  FSO,  ADO,  ENG 

30.  Determine  requirements  and  priorities  for  force  protection,  to  include  survivability  positions 
for  vehicles,  weapons,  systems,  and  equipment. 

XO,  S2,  S3,  S4,  FSO,  ENG 

3 1 .  Determine  air  defense  support  and  priorities. 

XO,  S2,  S3,  S4,  FSO,  ADO 

32.  Define  early  warning  requirements. 

52,  S3,  ADO 

33.  Determine  air  defense  decision  points. 

XO,  S2,  S3,  FSO,  ADO,  ENG 

34.  Determine  air  defense  movements  in  support  of  task  force  operations. 

53,  ADO 

35.  Determine  direct  and  indirect  fire  systems  in  an  air  defense  role. 

S2,  S3,  FSO,  ADO 

36.  Define  air  defense  fratricide  prevention  criteria. 

XO,  S3,  ADO 

37.  Determine  the  adequacy  of  the  area  for  CSS  operations. 

XO,  SI,  S2,  S3,  S4,  ENG 

38.  Determine  transportation  requirements  and  priorities. 

XO,  S3,  S4,  ENG 

39.  Determine  medical  support  requirements. 

XO,  SI,  S4 

40.  Identify  points  in  the  battle  when  surge  requirements  are  likely  to  be  generated. 
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XO,  S2,  S4 

41.  Determine  tactical  restrictions  on  CSS  operations. 

XO,  S3,  S4,  ENG 

42.  Compare  required  and  available  CSS  capability  to  identify  shortfalls  and  ways  and  means  to 
mitigate  the  effect  of  these  shortfalls. 

XO,  S2,  S3,  S4,  SIGO,  FSO,  ADO,  ENG 
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